-- Logs begin at Thu 2022-07-14 08:34:18 UTC, end at Thu 2022-07-14 09:36:36 UTC. -- Jul 14 09:03:21.635631 dv-cinder-849626-4-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jul 14 09:03:21.644626 dv-cinder-849626-4-iscsi devstack@keystone.service[80370]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jul 14 09:03:21.646796 dv-cinder-849626-4-iscsi devstack@keystone.service[80370]: *** Starting uWSGI 2.0.18-debian (64bit) on [Thu Jul 14 09:03:21 2022] *** Jul 14 09:03:21.646796 dv-cinder-849626-4-iscsi devstack@keystone.service[80370]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jul 14 09:03:21.646796 dv-cinder-849626-4-iscsi devstack@keystone.service[80370]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jul 14 09:03:21.646796 dv-cinder-849626-4-iscsi devstack@keystone.service[80370]: nodename: dv-cinder-849626-4-iscsi Jul 14 09:03:21.646796 dv-cinder-849626-4-iscsi devstack@keystone.service[80370]: machine: x86_64 Jul 14 09:03:21.646796 dv-cinder-849626-4-iscsi devstack@keystone.service[80370]: clock source: unix Jul 14 09:03:21.646796 dv-cinder-849626-4-iscsi devstack@keystone.service[80370]: pcre jit disabled Jul 14 09:03:21.646796 dv-cinder-849626-4-iscsi devstack@keystone.service[80370]: detected number of CPU cores: 4 Jul 14 09:03:21.646796 dv-cinder-849626-4-iscsi devstack@keystone.service[80370]: current working directory: / Jul 14 09:03:21.646796 dv-cinder-849626-4-iscsi devstack@keystone.service[80370]: detected binary path: /usr/bin/uwsgi-core Jul 14 09:03:21.646796 dv-cinder-849626-4-iscsi devstack@keystone.service[80370]: your processes number limit is 47680 Jul 14 09:03:21.646796 dv-cinder-849626-4-iscsi devstack@keystone.service[80370]: your memory page size is 4096 bytes Jul 14 09:03:21.646796 dv-cinder-849626-4-iscsi devstack@keystone.service[80370]: detected max file descriptor number: 2048 Jul 14 09:03:21.646796 dv-cinder-849626-4-iscsi devstack@keystone.service[80370]: lock engine: pthread robust mutexes Jul 14 09:03:21.647570 dv-cinder-849626-4-iscsi devstack@keystone.service[80370]: thunder lock: enabled Jul 14 09:03:21.647570 dv-cinder-849626-4-iscsi devstack@keystone.service[80370]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jul 14 09:03:21.647570 dv-cinder-849626-4-iscsi devstack@keystone.service[80370]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] Jul 14 09:03:21.690471 dv-cinder-849626-4-iscsi devstack@keystone.service[80370]: Python main interpreter initialized at 0x561c133ee9c0 Jul 14 09:03:21.690471 dv-cinder-849626-4-iscsi devstack@keystone.service[80370]: python threads support enabled Jul 14 09:03:21.690471 dv-cinder-849626-4-iscsi devstack@keystone.service[80370]: your server socket listen backlog is limited to 100 connections Jul 14 09:03:21.690471 dv-cinder-849626-4-iscsi devstack@keystone.service[80370]: your mercy for graceful operations on workers is 90 seconds Jul 14 09:03:21.690819 dv-cinder-849626-4-iscsi devstack@keystone.service[80370]: mapped 403077 bytes (393 KB) for 2 cores Jul 14 09:03:21.690891 dv-cinder-849626-4-iscsi devstack@keystone.service[80370]: *** Operational MODE: preforking *** Jul 14 09:03:21.691150 dv-cinder-849626-4-iscsi devstack@keystone.service[80370]: *** uWSGI is running in multiple interpreter mode *** Jul 14 09:03:21.691150 dv-cinder-849626-4-iscsi devstack@keystone.service[80370]: spawned uWSGI master process (pid: 80370) Jul 14 09:03:21.691392 dv-cinder-849626-4-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jul 14 09:03:21.692212 dv-cinder-849626-4-iscsi devstack@keystone.service[80370]: spawned uWSGI worker 1 (pid: 80381, cores: 1) Jul 14 09:03:21.693006 dv-cinder-849626-4-iscsi devstack@keystone.service[80370]: spawned uWSGI worker 2 (pid: 80382, cores: 1) Jul 14 09:03:21.693006 dv-cinder-849626-4-iscsi devstack@keystone.service[80370]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jul 14 09:03:27.040756 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 14 09:03:27.040756 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: from cryptography.utils import int_from_bytes Jul 14 09:03:27.041582 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 14 09:03:27.041582 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: from cryptography.utils import int_from_bytes Jul 14 09:03:27.042784 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 14 09:03:27.042784 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: from cryptography.utils import int_from_bytes Jul 14 09:03:27.043331 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 14 09:03:27.043331 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: from cryptography.utils import int_from_bytes Jul 14 09:03:27.089835 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 14 09:03:27.090030 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 14 09:03:27.090164 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] command line args: [] {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 14 09:03:27.090464 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 14 09:03:27.090595 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] ================================================================================ {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 14 09:03:27.090872 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 14 09:03:27.091014 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] admin_token = **** {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 09:03:27.091096 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 14 09:03:27.091162 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] config_dir = [] {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 09:03:27.091264 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] command line args: [] {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 14 09:03:27.091331 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 09:03:27.091477 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] config_source = [] {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 09:03:27.091568 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 14 09:03:27.091634 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] debug = True {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 09:03:27.091692 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] ================================================================================ {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 14 09:03:27.091822 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 09:03:27.092077 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] admin_token = **** {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 09:03:27.092149 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] config_dir = [] {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 09:03:27.092212 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 09:03:27.092336 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 09:03:27.092411 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 09:03:27.092477 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] config_source = [] {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 09:03:27.092587 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 09:03:27.092660 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] debug = True {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 09:03:27.092856 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 09:03:27.092971 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 09:03:27.093070 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] list_limit = None {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 09:03:27.093146 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 09:03:27.093211 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] log_config_append = None {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 09:03:27.093273 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 09:03:27.093425 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 09:03:27.093528 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 09:03:27.093629 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] log_dir = None {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 09:03:27.093732 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 09:03:27.093834 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] log_file = None {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 09:03:27.093928 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] list_limit = None {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 09:03:27.094020 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 09:03:27.094132 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] log_config_append = None {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 09:03:27.094217 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 09:03:27.094296 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 09:03:27.094412 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 09:03:27.094485 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] log_dir = None {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 09:03:27.094597 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] log_file = None {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 09:03:27.094676 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 09:03:27.094830 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 09:03:27.095016 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 09:03:27.095106 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 09:03:27.095251 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 09:03:27.095328 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 09:03:27.095456 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 09:03:27.095575 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 09:03:27.095660 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 09:03:27.095785 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 09:03:27.095866 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 09:03:27.095866 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 09:03:27.096048 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 09:03:27.096170 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 09:03:27.096246 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 09:03:27.096360 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 09:03:27.096441 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 09:03:27.096555 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 09:03:27.096628 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 09:03:27.096752 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 09:03:27.096836 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 09:03:27.096997 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 09:03:27.097089 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 09:03:27.097248 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 09:03:27.097493 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 09:03:27.097628 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.26/identity {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 09:03:27.097709 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] publish_errors = False {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 09:03:27.097786 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 09:03:27.097919 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 09:03:27.098082 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 09:03:27.098165 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 09:03:27.098365 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 09:03:27.098485 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.26/identity {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 09:03:27.098631 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 09:03:27.098722 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] publish_errors = False {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 09:03:27.098805 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 09:03:27.098929 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 09:03:27.099005 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] standard_threads = False {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 09:03:27.099097 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 09:03:27.099285 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 09:03:27.099409 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 09:03:27.099488 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 09:03:27.099559 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 09:03:27.099644 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 09:03:27.099770 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 09:03:27.099844 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] use_journal = False {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 09:03:27.099997 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] standard_threads = False {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 09:03:27.100131 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] use_json = False {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 09:03:27.100131 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] use_stderr = False {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 09:03:27.100279 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 09:03:27.100412 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] use_syslog = False {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 09:03:27.100487 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 09:03:27.100564 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 09:03:27.100635 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 09:03:27.100785 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] use_journal = False {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 09:03:27.100920 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.101008 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] use_json = False {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 09:03:27.101081 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.101158 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] use_stderr = False {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 09:03:27.101235 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.101305 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] use_syslog = False {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 09:03:27.101496 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.101627 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 09:03:27.101773 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.101900 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.102021 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.102141 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.102266 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.102381 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.102492 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.102603 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.102713 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.102828 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.102951 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.103069 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.103180 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.103323 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.103436 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.103552 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.103673 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.103785 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.103902 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.104013 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.104131 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.104242 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.104352 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.104474 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.104585 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.104702 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.104813 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.104923 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.105112 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.105225 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.105335 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.105445 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.105563 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.105673 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.105793 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.105906 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.106016 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.106132 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.106243 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.106354 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.106472 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.106595 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.106707 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.106818 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.106928 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.107038 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.107161 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.107312 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.107434 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.107546 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.107656 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.107775 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.107887 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.108005 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.108115 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.108226 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.108335 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.108456 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.108573 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.108691 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.108802 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.108913 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.109022 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.109144 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.109255 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.109365 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.109474 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.109591 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.109701 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.109819 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.109933 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.110044 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.110154 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.110264 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.110374 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.110567 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.110681 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.110798 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.110910 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.111027 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.111138 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.111273 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.111386 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.111499 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.111609 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.111719 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.111828 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.111945 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.112057 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.112167 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.112276 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.112386 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.112495 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.112613 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.112724 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.112820 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.112914 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.113006 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.113100 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.113202 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.113299 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.113394 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.113494 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.113660 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.113837 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.113969 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.114083 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.114196 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.114307 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.114424 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.114536 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.114656 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.114774 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.114886 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.114997 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.115108 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.115252 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.115379 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.115491 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.115595 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.115713 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.115826 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.115928 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.116044 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.116163 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.116273 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.116390 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.116511 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.116624 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.116741 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.116860 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.116981 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.117093 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.117201 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.117313 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.117423 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.117533 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.117652 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.117765 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.117875 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.117875 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.118083 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.118202 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.118313 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.118425 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.118535 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.118645 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.118755 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.118873 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.118986 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.118986 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.119271 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.119388 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.119498 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.119625 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.119740 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.119850 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.119961 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.120071 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.120182 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.120300 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.120418 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.120534 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.120648 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.120758 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.120869 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.120987 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.121099 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.121217 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.121329 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.121440 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.121552 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.121742 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.121857 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.121969 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.122080 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.122196 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.122308 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.122426 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.122549 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.122672 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.122785 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.122896 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.123006 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.123119 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.123244 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.123368 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.123481 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.123599 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.123711 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.123830 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.123949 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.124069 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.124181 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.124181 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.124390 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.124500 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.124611 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.124731 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.124843 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.124954 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.125064 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] auth.external = None {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.125174 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.125292 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.125405 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.125516 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.125628 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.125739 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.125850 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] auth.external = None {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.125994 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] auth.password = None {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.126116 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.126241 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] auth.token = None {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.126361 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.126475 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.126587 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.126698 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.126818 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] auth.password = None {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.126946 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.127060 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] auth.token = None {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.127171 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.127300 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.127412 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.127524 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.127652 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.127767 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.127879 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.127990 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.128108 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] credential.caching = True {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.128221 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.128340 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.128454 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.128566 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.128678 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.128789 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.128901 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] credential.caching = True {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.129020 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.129133 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.129245 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.129444 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.129444 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.129660 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.129781 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.129895 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.130007 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.130007 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.130216 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.130328 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.130447 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.130563 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.130675 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.130916 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.131089 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.131283 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.131417 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.131541 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.131691 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.131815 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.131958 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.132077 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.132199 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.132414 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] federation.caching = True {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.132584 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.132709 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.132899 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.133070 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.133210 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.133329 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] federation.caching = True {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.133442 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.133555 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.133771 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.133944 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.134083 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.134202 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.134317 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.134521 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.134691 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.134817 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.134948 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.135065 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.135180 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] identity.caching = True {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.135320 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.135443 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.135556 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.135684 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.135808 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.135923 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.136046 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.136160 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.136355 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.136472 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.136585 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.136698 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.136810 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.136923 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.137043 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] identity.caching = True {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.137158 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.137281 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.137395 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.137508 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.137621 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.137741 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.137856 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.137968 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.138081 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.138193 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.138306 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.138427 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.138541 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.138659 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.138773 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.138885 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.138999 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.139120 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.139249 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.139365 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.139477 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.139590 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.139714 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.139838 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.139958 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.140072 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.140185 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.140297 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.140410 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.140531 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.140645 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.140757 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.140882 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.140882 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.141095 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.141279 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.141457 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.141595 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.141702 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.141884 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.141884 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.142128 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.142283 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.142399 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.142520 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.142667 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.142780 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.142903 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.143017 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.143131 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.143263 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.143377 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.143490 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.143617 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.143732 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.143844 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.143957 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.144069 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.144181 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.144302 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.144416 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.144530 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.144643 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.144755 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.144931 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.145061 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.145175 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.145294 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.145408 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.145521 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.145634 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.145757 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.145872 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.145984 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.146096 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.146209 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.146321 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.146442 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.146556 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.146676 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.146789 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.146901 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.147013 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.147133 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] ldap.user = None {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.147307 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.147421 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.147534 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.147646 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.147760 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.147886 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.148001 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.148113 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.148225 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.148343 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.148464 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.148578 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.148691 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.148805 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.148916 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.149028 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.149148 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.149271 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.149387 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.149500 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.149612 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] ldap.user = None {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.149724 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.149851 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.149966 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.150077 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.150198 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.150311 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.150423 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.150543 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.150657 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.150770 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.150883 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.151001 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.151112 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.151249 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.151365 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.151477 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.151589 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.151700 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.151813 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.151934 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.152048 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.152160 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.152272 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.152384 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.152502 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.152623 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.152737 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.152868 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.152984 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.153096 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.153208 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.153328 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.153443 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.153555 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.153667 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.153779 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.153900 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.154023 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.154137 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.154250 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.154362 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.154482 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.154594 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.154718 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.154842 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.154956 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.155069 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.155182 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.155314 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.155439 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.155554 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.155667 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.155779 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] resource.caching = True {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.155891 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.156004 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.156126 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.156240 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.156353 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.156466 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.156578 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.156690 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.156811 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.156925 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.157039 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.157153 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.157265 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.157377 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.157498 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.157613 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.157725 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.157837 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.157949 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.158061 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] role.caching = True {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.158182 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] resource.caching = True {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.158297 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] role.driver = sql {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.158410 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.158522 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.158641 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.158755 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.158939 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.159055 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.159168 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.159335 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.159448 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.159560 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.159686 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.159801 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.159914 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.160026 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.160138 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.160250 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.160371 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.160535 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.160652 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] role.caching = True {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.160764 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.160877 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] role.driver = sql {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.160993 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.161115 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.161115 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.161328 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.161442 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.161555 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.161676 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.161790 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.161902 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.161902 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.162114 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.162226 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.162346 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.162461 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.162574 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.162687 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.162799 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.162912 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.163032 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.163145 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.163277 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.163389 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.163502 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.163614 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.163740 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.163854 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.163966 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.163966 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.164176 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.164288 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.164408 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.164523 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.164635 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.164747 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.164858 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.164970 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.165090 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.165204 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.165316 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.165428 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.165541 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.165653 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.165772 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.165877 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.165989 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.166101 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.166212 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.166332 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] token.caching = True {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.166444 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.166556 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.166668 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.166782 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.166893 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.167013 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.167126 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.167253 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.167367 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.167479 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.167590 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.167715 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.167829 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.167941 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.168054 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.168166 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.168278 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.168399 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.168512 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] token.caching = True {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.168624 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.168737 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.168848 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.168960 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.169080 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.169193 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.169305 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.169417 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.169528 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.169641 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.169762 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.169875 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.169995 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.170108 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.170220 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.170331 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.170451 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] database.connection = **** {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.170573 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.170698 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.170813 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.170929 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.171041 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.171153 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.171305 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.171431 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.171545 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.171657 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.171768 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.171882 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.171993 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.172113 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.172227 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.172340 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.172451 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.172564 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.172676 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] database.connection = **** {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.172800 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.172931 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.173046 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.173232 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.173349 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.173461 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.173573 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.173694 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.173808 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.173922 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.174035 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.174147 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.174259 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.174379 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.174492 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.174605 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.174716 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.174828 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.174940 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 14 09:03:27.175062 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.175062 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.175291 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.175402 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.175639 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.175882 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.176182 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.176552 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.176792 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:03:27.176963 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 14 09:03:27.205608 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80382) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 09:03:27.205847 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80382) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 09:03:27.206063 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80382) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 09:03:27.206521 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80381) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 09:03:27.206757 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80381) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 09:03:27.206983 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80381) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 09:03:27.214476 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80382) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 09:03:27.215427 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80381) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 09:03:27.242522 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80382) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 09:03:27.242710 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80382) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 09:03:27.243528 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80381) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 09:03:27.243762 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80381) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 09:03:27.384725 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80382) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 09:03:27.385034 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80381) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 09:03:27.433366 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80382) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 09:03:27.433572 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80382) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 09:03:27.434125 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80381) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 09:03:27.434361 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80381) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 09:03:27.497593 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80382) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 09:03:27.497788 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80382) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 09:03:27.497898 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80381) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 09:03:27.498017 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80382) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 09:03:27.498120 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80381) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 09:03:27.498231 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80382) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 09:03:27.498317 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80381) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 09:03:27.498511 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80382) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 09:03:27.498636 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80381) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 09:03:27.498715 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80382) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 09:03:27.498808 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80381) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 09:03:27.499003 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80381) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 09:03:27.523229 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80382) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 09:03:27.523493 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80382) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 09:03:27.523618 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80381) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 09:03:27.523744 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80382) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 09:03:27.523824 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80381) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 09:03:27.523989 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80382) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 09:03:27.524115 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80381) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 09:03:27.524193 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80381) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 09:03:27.524275 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80382) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 09:03:27.524378 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80382) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 09:03:27.524460 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80381) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 09:03:27.524660 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80382) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 09:03:27.524785 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80381) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 09:03:27.524785 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80381) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 09:03:27.535819 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80382) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 09:03:27.536112 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80381) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 09:03:27.548565 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80382) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 09:03:27.548795 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80382) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 09:03:27.549016 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80382) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 09:03:27.550987 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80381) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 09:03:27.552767 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80382) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 09:03:27.552910 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80381) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 09:03:27.553235 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80382) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 09:03:27.553334 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80381) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 09:03:27.553402 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80382) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 09:03:27.553484 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80381) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 09:03:27.553549 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80382) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 09:03:27.553633 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80381) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 09:03:27.553633 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80381) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 09:03:27.553633 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80381) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 09:03:27.569056 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:03:27.569269 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 14 09:03:27.569536 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:03:27.569744 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 14 09:03:27.569889 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:03:27.570006 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 14 09:03:27.570311 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:03:27.570437 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:03:27.570575 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 14 09:03:27.570655 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:03:27.570734 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:03:27.570809 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:03:27.570999 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:03:27.571126 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:03:27.571294 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:03:27.571539 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:03:27.581866 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:03:27.582143 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:03:27.582370 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:03:27.582462 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:03:27.582620 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:03:27.582761 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:03:27.582903 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:03:27.583129 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:03:27.590971 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80382) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 09:03:27.591435 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80381) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 09:03:27.594266 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80382) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 09:03:27.594545 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:03:27.594672 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80381) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 09:03:27.594812 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:03:27.595011 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:03:27.595149 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:03:27.595318 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:03:27.595412 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:03:27.595534 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:03:27.595615 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:03:27.595814 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:03:27.595940 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:03:27.596080 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:03:27.596162 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:03:27.596328 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:03:27.596455 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:03:27.596608 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:03:27.596691 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:03:27.596853 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:03:27.596980 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:03:27.597104 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:03:27.597380 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:03:27.623242 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:03:27.624055 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:03:27.624878 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80382) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 09:03:27.625163 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:03:27.625666 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80381) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 09:03:27.625947 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:03:27.630444 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80382) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 09:03:27.630724 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:03:27.631007 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:03:27.631225 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80381) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 09:03:27.631530 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:03:27.631812 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:03:27.638485 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80382) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 09:03:27.638711 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:03:27.639170 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80381) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 09:03:27.639480 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:03:27.643178 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80382) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 09:03:27.643489 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:03:27.643780 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:03:27.643914 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80381) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 09:03:27.644015 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:03:27.644181 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:03:27.644277 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:03:27.644484 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:03:27.644611 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:03:27.644871 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:03:27.645014 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:03:27.645165 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:03:27.645293 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:03:27.645432 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:03:27.645668 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:03:27.645935 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:03:27.663713 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:03:27.663983 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:03:27.664469 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:03:27.664747 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:03:27.668601 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80382) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 09:03:27.669377 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80381) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 09:03:27.672262 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:03:27.672559 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:03:27.673023 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:03:27.673321 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:03:27.677736 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80382) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 09:03:27.678482 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80381) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 09:03:27.681379 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80382) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 09:03:27.682118 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80381) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 09:03:27.685042 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:03:27.685340 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:03:27.685631 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:03:27.685756 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:03:27.685894 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:03:27.686051 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:03:27.686195 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:03:27.686332 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:03:27.686413 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:03:27.686656 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:03:27.686935 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:03:27.687227 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:03:27.705789 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80382) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 09:03:27.706073 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:03:27.706339 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:03:27.706540 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80381) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 09:03:27.706658 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:03:27.706790 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:03:27.707063 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:03:27.707406 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:03:27.714581 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:03:27.715333 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:03:27.716315 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:03:27.716591 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:03:27.717036 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:03:27.717308 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:03:27.719790 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80382) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 09:03:27.720071 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:03:27.720350 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:03:27.720573 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80381) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 09:03:27.720701 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:03:27.720839 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:03:27.720932 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:03:27.721124 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:03:27.721425 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:03:27.721717 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:03:27.734129 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80382) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 09:03:27.734407 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:03:27.734697 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:03:27.734942 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80381) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 09:03:27.735067 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:03:27.735250 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:03:27.735335 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:03:27.735518 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:03:27.735643 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:03:27.735780 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:03:27.735859 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:03:27.736079 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:03:27.736366 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:03:27.736643 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:03:27.755221 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80382) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 09:03:27.755786 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80381) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 09:03:27.758563 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80382) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 09:03:27.759038 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80381) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 09:03:27.761889 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:03:27.762375 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:03:27.763513 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:03:27.763997 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:03:27.765288 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80382) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 09:03:27.765566 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:03:27.765771 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80381) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 09:03:27.765899 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:03:27.766039 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:03:27.766330 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:03:27.773327 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:03:27.773789 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:03:27.774937 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80382) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 09:03:27.775437 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80381) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 09:03:27.778179 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:03:27.778468 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:03:27.778631 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:03:27.778730 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:03:27.778939 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:03:27.779080 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:03:27.779253 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:03:27.779485 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:03:27.788232 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80382) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 09:03:27.788517 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:03:27.788746 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80381) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 09:03:27.788892 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:03:27.789032 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:03:27.789293 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:03:27.796657 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80382) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 09:03:27.796941 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:03:27.797166 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80381) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 09:03:27.797294 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:03:27.797432 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:03:27.797513 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:03:27.797710 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:03:27.797838 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:03:27.797995 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:03:27.798079 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:03:27.798238 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:03:27.798365 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:03:27.798502 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:03:27.798597 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:03:27.798751 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:03:27.798844 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:03:27.799056 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:03:27.799274 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:03:27.799400 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:03:27.799491 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:03:27.799685 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:03:27.799813 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:03:27.799935 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:03:27.800028 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:03:27.800235 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:03:27.800361 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:03:27.800520 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:03:27.800798 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:03:27.838202 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x561c133ee9c0 pid: 80382 (default app) Jul 14 09:03:27.838792 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x561c133ee9c0 pid: 80381 (default app) Jul 14 09:03:27.841610 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef30f925-0428-4f5d-9ea5-5ead7306764b None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:03:27.841903 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef30f925-0428-4f5d-9ea5-5ead7306764b None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:03:27.842188 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef30f925-0428-4f5d-9ea5-5ead7306764b None None] PATH_INFO: `/v3` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:03:27.846505 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1/1] 10.222.0.26 () {50 vars in 791 bytes} [Thu Jul 14 09:03:27 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:03:35.465536 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c566bb36-b913-4c03-bb0d-6ac3c5c8e9db None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:03:35.466435 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c566bb36-b913-4c03-bb0d-6ac3c5c8e9db None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:03:35.466435 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c566bb36-b913-4c03-bb0d-6ac3c5c8e9db None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:03:35.472845 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1/2] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:03:35 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:03:35.478721 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567102cc-b092-467c-9e6b-e4b87e2bcee7 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:03:35.479155 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567102cc-b092-467c-9e6b-e4b87e2bcee7 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:03:35.479272 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567102cc-b092-467c-9e6b-e4b87e2bcee7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:03:35.548089 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: INFO dbcounter [None req-567102cc-b092-467c-9e6b-e4b87e2bcee7 None None] Registered counter for database keystone Jul 14 09:03:35.570567 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG oslo_db.sqlalchemy.engines [None req-567102cc-b092-467c-9e6b-e4b87e2bcee7 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80382) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 14 09:03:35.577732 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writer thread running {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 14 09:03:35.769897 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG passlib.handlers.bcrypt [None req-567102cc-b092-467c-9e6b-e4b87e2bcee7 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80382) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 14 09:03:35.770315 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG passlib.handlers.bcrypt [None req-567102cc-b092-467c-9e6b-e4b87e2bcee7 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80382) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 14 09:03:35.825357 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-567102cc-b092-467c-9e6b-e4b87e2bcee7 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:03:35.826964 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-567102cc-b092-467c-9e6b-e4b87e2bcee7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:03:35.951950 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 2/3] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:03:35 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 474 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Jul 14 09:03:36.188166 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c566bb36-b913-4c03-bb0d-6ac3c5c8e9db None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:03:36.260260 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: INFO dbcounter [None req-c566bb36-b913-4c03-bb0d-6ac3c5c8e9db None None] Registered counter for database keystone Jul 14 09:03:36.281800 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG oslo_db.sqlalchemy.engines [None req-c566bb36-b913-4c03-bb0d-6ac3c5c8e9db None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80381) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 14 09:03:36.292951 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writer thread running {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 14 09:03:36.420360 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c566bb36-b913-4c03-bb0d-6ac3c5c8e9db None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:03:36.425847 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63ec43b-ae53-4629-ab7c-6e0f2e259317 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:03:36.426517 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63ec43b-ae53-4629-ab7c-6e0f2e259317 admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:03:36.426799 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63ec43b-ae53-4629-ab7c-6e0f2e259317 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:03:36.427075 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63ec43b-ae53-4629-ab7c-6e0f2e259317 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:03:36.436697 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63ec43b-ae53-4629-ab7c-6e0f2e259317 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:03:36.487421 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: WARNING keystone.common.rbac_enforcer.enforcer [None req-f63ec43b-ae53-4629-ab7c-6e0f2e259317 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 14 09:03:36.488015 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63ec43b-ae53-4629-ab7c-6e0f2e259317 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:03:36.499125 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 2/4] 10.222.0.26 () {58 vars in 1324 bytes} [Thu Jul 14 09:03:36 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 14 09:03:36.509161 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567102cc-b092-467c-9e6b-e4b87e2bcee7 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:03:36.527596 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567102cc-b092-467c-9e6b-e4b87e2bcee7 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:03:36.533216 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86b280f-2cac-42ef-a869-768a617e0f53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:03:36.534419 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86b280f-2cac-42ef-a869-768a617e0f53 admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:03:36.534709 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86b280f-2cac-42ef-a869-768a617e0f53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:03:36.534988 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86b280f-2cac-42ef-a869-768a617e0f53 admin admin] PATH_INFO: `/v3/projects/2fd144ddf49f41d58d6674bed415082b` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:03:36.536725 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86b280f-2cac-42ef-a869-768a617e0f53 admin admin] RBAC: Authorizing `identity:get_project(project_id=2fd144ddf49f41d58d6674bed415082b)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:03:36.589558 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: WARNING keystone.common.rbac_enforcer.enforcer [None req-d86b280f-2cac-42ef-a869-768a617e0f53 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 14 09:03:36.589898 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86b280f-2cac-42ef-a869-768a617e0f53 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:03:36.594584 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 3/5] 10.222.0.26 () {56 vars in 1207 bytes} [Thu Jul 14 09:03:36 2022] GET /identity/v3/projects/2fd144ddf49f41d58d6674bed415082b => generated 344 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:03:38.614174 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e16076-1c24-4766-a48a-c590e8c61627 None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:03:38.614993 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e16076-1c24-4766-a48a-c590e8c61627 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:03:38.614993 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e16076-1c24-4766-a48a-c590e8c61627 None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:03:38.621887 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 3/6] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:03:38 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:03:38.627714 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213fd89b-0bf7-4931-aae3-e2d3798e8604 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:03:38.628142 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213fd89b-0bf7-4931-aae3-e2d3798e8604 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:03:38.628236 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213fd89b-0bf7-4931-aae3-e2d3798e8604 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:03:38.649721 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-213fd89b-0bf7-4931-aae3-e2d3798e8604 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:03:38.651255 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-213fd89b-0bf7-4931-aae3-e2d3798e8604 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:03:38.661458 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 4/7] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:03:38 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 14 09:03:38.683869 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e16076-1c24-4766-a48a-c590e8c61627 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:03:38.700289 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e16076-1c24-4766-a48a-c590e8c61627 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:03:38.706422 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04705726-5549-4339-817e-11c99f9428c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:03:38.707049 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04705726-5549-4339-817e-11c99f9428c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:03:38.707365 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04705726-5549-4339-817e-11c99f9428c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:03:38.707644 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04705726-5549-4339-817e-11c99f9428c3 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:03:38.715823 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04705726-5549-4339-817e-11c99f9428c3 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:03:38.717616 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04705726-5549-4339-817e-11c99f9428c3 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:03:38.726706 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 4/8] 10.222.0.26 () {58 vars in 1324 bytes} [Thu Jul 14 09:03:38 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 14 09:03:38.734885 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213fd89b-0bf7-4931-aae3-e2d3798e8604 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:03:38.749554 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213fd89b-0bf7-4931-aae3-e2d3798e8604 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:03:38.755227 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a54fe3-a618-4fd5-8392-98d3d0677b65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:03:38.756084 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a54fe3-a618-4fd5-8392-98d3d0677b65 admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:03:38.756362 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a54fe3-a618-4fd5-8392-98d3d0677b65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:03:38.756639 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a54fe3-a618-4fd5-8392-98d3d0677b65 admin admin] PATH_INFO: `/v3/users/d00c29cc14a043099fbeda47a3389faa` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:03:38.758074 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a54fe3-a618-4fd5-8392-98d3d0677b65 admin admin] RBAC: Authorizing `identity:get_user(user_id=d00c29cc14a043099fbeda47a3389faa)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:03:38.759922 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a54fe3-a618-4fd5-8392-98d3d0677b65 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:03:38.761493 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 5/9] 10.222.0.26 () {56 vars in 1198 bytes} [Thu Jul 14 09:03:38 2022] GET /identity/v3/users/d00c29cc14a043099fbeda47a3389faa => generated 247 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:03:40.807637 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a87b0ea-197f-4257-8253-a82bcb4baca2 None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:03:40.808157 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a87b0ea-197f-4257-8253-a82bcb4baca2 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:03:40.808157 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a87b0ea-197f-4257-8253-a82bcb4baca2 None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:03:40.809457 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 5/10] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:03:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:03:40.814989 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904be65f-b0d0-41ea-a874-b44ecab50528 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:03:40.815437 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904be65f-b0d0-41ea-a874-b44ecab50528 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:03:40.815661 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904be65f-b0d0-41ea-a874-b44ecab50528 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:03:40.833093 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-904be65f-b0d0-41ea-a874-b44ecab50528 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:03:40.834264 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-904be65f-b0d0-41ea-a874-b44ecab50528 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:03:40.845695 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e217bf3-1bc4-4887-88a6-b56655b2287e None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:03:40.845951 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e217bf3-1bc4-4887-88a6-b56655b2287e None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:03:40.846428 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e217bf3-1bc4-4887-88a6-b56655b2287e None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:03:40.847823 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 6/11] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:03:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 09:03:40.852536 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3205abcf-5706-46d9-8226-9adcea46b2de None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:03:40.852820 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3205abcf-5706-46d9-8226-9adcea46b2de None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:03:40.853077 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3205abcf-5706-46d9-8226-9adcea46b2de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:03:40.892385 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG passlib.handlers.bcrypt [None req-3205abcf-5706-46d9-8226-9adcea46b2de None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80381) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 14 09:03:40.892795 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG passlib.handlers.bcrypt [None req-3205abcf-5706-46d9-8226-9adcea46b2de None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80381) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 14 09:03:40.919075 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 6/12] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:03:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 09:03:40.938073 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-3205abcf-5706-46d9-8226-9adcea46b2de None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:03:40.939724 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-3205abcf-5706-46d9-8226-9adcea46b2de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:03:40.942838 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904be65f-b0d0-41ea-a874-b44ecab50528 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:03:40.967778 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:03:41.040477 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 7/13] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:03:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 09:03:41.049840 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904be65f-b0d0-41ea-a874-b44ecab50528 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:03:41.062865 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3205abcf-5706-46d9-8226-9adcea46b2de None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:03:41.083269 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c679aa39-6679-4857-933d-d6738f11e7e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:03:41.084071 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c679aa39-6679-4857-933d-d6738f11e7e3 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:03:41.084356 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c679aa39-6679-4857-933d-d6738f11e7e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:03:41.084644 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c679aa39-6679-4857-933d-d6738f11e7e3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:03:41.096451 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c679aa39-6679-4857-933d-d6738f11e7e3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 09:03:41.110467 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-c679aa39-6679-4857-933d-d6738f11e7e3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 09:03:41.110467 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:03:41.110467 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:03:41.110467 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:03:41.110467 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:03:41.110467 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:03:41.110467 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:03:41.110467 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:03:41.110467 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:03:41.110467 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:03:41.110467 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:03:41.110467 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:03:41.110467 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 09:03:41.110467 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 09:03:41.110467 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 09:03:41.110467 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 09:03:41.111632 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 09:03:41.111632 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 09:03:41.111632 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 09:03:41.111632 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 09:03:41.111632 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:03:41.111632 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:03:41.111632 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:03:41.111632 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:03:41.111632 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:03:41.111632 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:03:41.111632 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:03:41.111632 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 14 09:03:41.111632 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:03:41.111632 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:03:41.111632 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:03:41.111632 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:03:41.112484 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:03:41.112484 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:03:41.112484 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:03:41.112484 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:03:41.112484 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 09:03:41.112484 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 09:03:41.112484 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 09:03:41.112484 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 09:03:41.112484 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 09:03:41.112484 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 09:03:41.112484 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 09:03:41.112484 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 14 09:03:41.112484 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 7/14] 10.222.0.26 () {56 vars in 1096 bytes} [Thu Jul 14 09:03:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:03:41.117946 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c679aa39-6679-4857-933d-d6738f11e7e3 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:03:41.167203 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3205abcf-5706-46d9-8226-9adcea46b2de None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:03:41.169508 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:03:41.202001 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:03:41.205429 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b0097a-3d39-4c84-8fab-73ca74de4249 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:03:41.206346 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b0097a-3d39-4c84-8fab-73ca74de4249 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:03:41.206627 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b0097a-3d39-4c84-8fab-73ca74de4249 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:03:41.206899 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b0097a-3d39-4c84-8fab-73ca74de4249 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:03:41.220999 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c679aa39-6679-4857-933d-d6738f11e7e3 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:03:41.255608 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc34b7a-8374-43c9-bda8-394f06201c5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:03:41.256455 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc34b7a-8374-43c9-bda8-394f06201c5a None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:03:41.256742 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc34b7a-8374-43c9-bda8-394f06201c5a None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:03:41.257311 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc34b7a-8374-43c9-bda8-394f06201c5a None admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:03:41.258083 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc34b7a-8374-43c9-bda8-394f06201c5a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:03:41.258394 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc34b7a-8374-43c9-bda8-394f06201c5a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:03:41.259859 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc34b7a-8374-43c9-bda8-394f06201c5a None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:03:41.272028 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 8/15] 10.222.0.26 () {56 vars in 1099 bytes} [Thu Jul 14 09:03:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:03:41.275594 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b0097a-3d39-4c84-8fab-73ca74de4249 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:03:41.277115 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b0097a-3d39-4c84-8fab-73ca74de4249 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:03:41.277652 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc34b7a-8374-43c9-bda8-394f06201c5a None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:03:41.377764 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 8/16] 10.222.0.26 () {58 vars in 1282 bytes} [Thu Jul 14 09:03:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 09:03:41.380139 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc34b7a-8374-43c9-bda8-394f06201c5a None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:03:41.385781 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b0097a-3d39-4c84-8fab-73ca74de4249 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:03:41.388608 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:03:41.416928 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93be6a31-43a7-4c85-abf1-3ed65bde4250 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:03:41.417802 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93be6a31-43a7-4c85-abf1-3ed65bde4250 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:03:41.418088 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93be6a31-43a7-4c85-abf1-3ed65bde4250 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:03:41.418357 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93be6a31-43a7-4c85-abf1-3ed65bde4250 None admin] PATH_INFO: `/v3/users/d00c29cc14a043099fbeda47a3389faa` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:03:41.419612 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93be6a31-43a7-4c85-abf1-3ed65bde4250 None admin] RBAC: Authorizing `identity:get_user(user_id=d00c29cc14a043099fbeda47a3389faa)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:03:41.421087 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93be6a31-43a7-4c85-abf1-3ed65bde4250 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:03:41.422888 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 9/17] 10.222.0.26 () {56 vars in 1177 bytes} [Thu Jul 14 09:03:41 2022] GET /identity/v3/users/d00c29cc14a043099fbeda47a3389faa => generated 247 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:03:41.428406 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93be6a31-43a7-4c85-abf1-3ed65bde4250 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:03:41.444160 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:03:41.487158 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b0097a-3d39-4c84-8fab-73ca74de4249 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:03:41.521008 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779f1782-f594-4678-a4b4-28b474bc8222 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:03:41.521803 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779f1782-f594-4678-a4b4-28b474bc8222 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:03:41.522093 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779f1782-f594-4678-a4b4-28b474bc8222 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:03:41.522369 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779f1782-f594-4678-a4b4-28b474bc8222 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:03:41.531918 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93be6a31-43a7-4c85-abf1-3ed65bde4250 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:03:41.540842 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779f1782-f594-4678-a4b4-28b474bc8222 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:03:41.542318 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779f1782-f594-4678-a4b4-28b474bc8222 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:03:41.564528 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-779f1782-f594-4678-a4b4-28b474bc8222 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 09:03:41.564528 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:03:41.564528 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:03:41.564528 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:03:41.564528 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:03:41.564528 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:03:41.564528 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:03:41.564528 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:03:41.564528 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:03:41.564528 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:03:41.564528 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:03:41.564528 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:03:41.564528 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 09:03:41.564528 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 09:03:41.564528 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 09:03:41.565341 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 09:03:41.565341 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:03:41.565341 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:03:41.565341 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:03:41.565341 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:03:41.565341 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:03:41.565341 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:03:41.565341 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:03:41.565341 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 14 09:03:41.565341 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:03:41.565341 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:03:41.565341 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:03:41.565341 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:03:41.565341 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:03:41.565341 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:03:41.565341 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:03:41.565341 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:03:41.566467 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 09:03:41.566467 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 09:03:41.566467 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 09:03:41.566467 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 14 09:03:41.566467 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 9/18] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jul 14 09:03:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 09:03:41.566901 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c9b9d4-907f-4d94-9dda-5c02abfa85ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:03:41.567677 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c9b9d4-907f-4d94-9dda-5c02abfa85ae None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:03:41.567960 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c9b9d4-907f-4d94-9dda-5c02abfa85ae None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:03:41.568234 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c9b9d4-907f-4d94-9dda-5c02abfa85ae None admin] PATH_INFO: `/v3/domains/default` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:03:41.569421 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c9b9d4-907f-4d94-9dda-5c02abfa85ae None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:03:41.570880 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c9b9d4-907f-4d94-9dda-5c02abfa85ae None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:03:41.571675 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779f1782-f594-4678-a4b4-28b474bc8222 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:03:41.572535 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 10/19] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jul 14 09:03:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 09:03:41.577914 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c9b9d4-907f-4d94-9dda-5c02abfa85ae None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:03:41.629001 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:03:41.658135 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:03:41.674187 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779f1782-f594-4678-a4b4-28b474bc8222 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:03:41.678230 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c9b9d4-907f-4d94-9dda-5c02abfa85ae None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:03:41.709148 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c827f3-803e-4582-be29-e1e504cb365b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:03:41.709813 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c827f3-803e-4582-be29-e1e504cb365b None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:03:41.710096 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c827f3-803e-4582-be29-e1e504cb365b None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:03:41.710370 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c827f3-803e-4582-be29-e1e504cb365b None admin] PATH_INFO: `/v3/domains` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:03:41.711101 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c827f3-803e-4582-be29-e1e504cb365b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:03:41.711441 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c827f3-803e-4582-be29-e1e504cb365b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:03:41.712186 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b428d2-2310-41c3-9c24-17f7f6a56aa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:03:41.712848 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c827f3-803e-4582-be29-e1e504cb365b None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:03:41.712946 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b428d2-2310-41c3-9c24-17f7f6a56aa3 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:03:41.713162 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b428d2-2310-41c3-9c24-17f7f6a56aa3 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:03:41.713443 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b428d2-2310-41c3-9c24-17f7f6a56aa3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:03:41.714283 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b428d2-2310-41c3-9c24-17f7f6a56aa3 None admin] RBAC: Adding query filter params (role.id=7b549d58d1724395b343609d895c80d0, scope.domain.id=default, user.id=d00c29cc14a043099fbeda47a3389faa) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:03:41.714598 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b428d2-2310-41c3-9c24-17f7f6a56aa3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:03:41.716083 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b428d2-2310-41c3-9c24-17f7f6a56aa3 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:03:41.722991 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 11/20] 10.222.0.26 () {56 vars in 1322 bytes} [Thu Jul 14 09:03:41 2022] GET /identity/v3/role_assignments?user.id=d00c29cc14a043099fbeda47a3389faa&scope.domain.id=default&role.id=7b549d58d1724395b343609d895c80d0 => generated 233 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 09:03:41.730837 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 10/21] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Jul 14 09:03:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 09:03:44.008910 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d84861-dfba-460f-981d-ee66f54910ef None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:03:44.009883 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d84861-dfba-460f-981d-ee66f54910ef None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:03:44.010383 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d84861-dfba-460f-981d-ee66f54910ef None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:03:44.012044 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 12/22] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:03:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:03:44.017484 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1b045a-1ad8-4d84-858f-00da23342706 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:03:44.018273 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1b045a-1ad8-4d84-858f-00da23342706 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:03:44.018769 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1b045a-1ad8-4d84-858f-00da23342706 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:03:44.044169 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-ba1b045a-1ad8-4d84-858f-00da23342706 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:03:44.045703 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-ba1b045a-1ad8-4d84-858f-00da23342706 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:03:44.133586 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 11/23] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:03:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 09:03:44.180698 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d84861-dfba-460f-981d-ee66f54910ef None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:03:44.256720 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d10dc85-549b-4045-9e77-583f12df5f75 None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:03:44.257059 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d10dc85-549b-4045-9e77-583f12df5f75 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:03:44.257353 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d10dc85-549b-4045-9e77-583f12df5f75 None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:03:44.259104 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 12/24] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:03:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 09:03:44.264563 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfd3d34-4483-4906-9750-ad140bed422f None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:03:44.264859 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfd3d34-4483-4906-9750-ad140bed422f None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:03:44.265159 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfd3d34-4483-4906-9750-ad140bed422f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:03:44.296914 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-fdfd3d34-4483-4906-9750-ad140bed422f None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:03:44.298310 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-fdfd3d34-4483-4906-9750-ad140bed422f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:03:44.351214 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d84861-dfba-460f-981d-ee66f54910ef None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:03:44.387564 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:03:44.410306 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfca1bac-3ec1-4826-8d74-a31ff4a8a994 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:03:44.411536 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfca1bac-3ec1-4826-8d74-a31ff4a8a994 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:03:44.412023 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfca1bac-3ec1-4826-8d74-a31ff4a8a994 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:03:44.412495 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfca1bac-3ec1-4826-8d74-a31ff4a8a994 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:03:44.429712 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfca1bac-3ec1-4826-8d74-a31ff4a8a994 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:03:44.439513 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 13/25] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:03:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 09:03:44.440863 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfca1bac-3ec1-4826-8d74-a31ff4a8a994 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:03:44.463669 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:03:44.465971 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfd3d34-4483-4906-9750-ad140bed422f None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:03:44.473935 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-dfca1bac-3ec1-4826-8d74-a31ff4a8a994 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 09:03:44.473935 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:03:44.473935 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:03:44.473935 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:03:44.473935 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:03:44.473935 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:03:44.473935 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:03:44.473935 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:03:44.473935 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:03:44.473935 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:03:44.473935 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:03:44.473935 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:03:44.473935 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 09:03:44.473935 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 09:03:44.473935 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 09:03:44.474897 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 09:03:44.474897 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:03:44.474897 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:03:44.474897 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:03:44.474897 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:03:44.474897 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:03:44.474897 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:03:44.474897 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:03:44.474897 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 14 09:03:44.474897 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:03:44.474897 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:03:44.474897 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:03:44.474897 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:03:44.474897 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:03:44.474897 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:03:44.474897 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:03:44.474897 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:03:44.475825 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 09:03:44.475825 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 09:03:44.475825 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 09:03:44.475825 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 14 09:03:44.477619 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 13/26] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jul 14 09:03:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:03:44.487775 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfca1bac-3ec1-4826-8d74-a31ff4a8a994 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:03:44.671283 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfca1bac-3ec1-4826-8d74-a31ff4a8a994 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:03:44.740949 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfd3d34-4483-4906-9750-ad140bed422f None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:03:44.746166 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c60bb94-20cc-4fbd-aab3-6b6b312b6e70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:03:44.746842 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c60bb94-20cc-4fbd-aab3-6b6b312b6e70 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:03:44.747128 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c60bb94-20cc-4fbd-aab3-6b6b312b6e70 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:03:44.747442 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c60bb94-20cc-4fbd-aab3-6b6b312b6e70 None admin] PATH_INFO: `/v3/domains` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:03:44.748189 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c60bb94-20cc-4fbd-aab3-6b6b312b6e70 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:03:44.748498 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c60bb94-20cc-4fbd-aab3-6b6b312b6e70 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:03:44.749955 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c60bb94-20cc-4fbd-aab3-6b6b312b6e70 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:03:44.787828 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 14/27] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Jul 14 09:03:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:03:44.789202 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b3b6c9-1582-47c4-b3a2-407b365e29a2 None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:03:44.789490 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b3b6c9-1582-47c4-b3a2-407b365e29a2 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:03:44.789766 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b3b6c9-1582-47c4-b3a2-407b365e29a2 None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:03:44.792577 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 15/28] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:03:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:03:44.796012 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b3b6c9-1582-47c4-b3a2-407b365e29a2 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:03:44.815348 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9277d5d7-086d-4683-92e4-dcd077f9d736 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:03:44.818316 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9277d5d7-086d-4683-92e4-dcd077f9d736 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:03:44.818763 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9277d5d7-086d-4683-92e4-dcd077f9d736 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:03:44.819182 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9277d5d7-086d-4683-92e4-dcd077f9d736 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:03:44.842543 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9277d5d7-086d-4683-92e4-dcd077f9d736 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 09:03:44.852733 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-9277d5d7-086d-4683-92e4-dcd077f9d736 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 09:03:44.852733 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:03:44.852733 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:03:44.852733 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:03:44.852733 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:03:44.852733 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:03:44.852733 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:03:44.852733 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:03:44.852733 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:03:44.852733 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:03:44.852733 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:03:44.852733 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:03:44.852733 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 09:03:44.852733 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 09:03:44.852733 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 09:03:44.852733 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 09:03:44.853773 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 09:03:44.853773 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 09:03:44.853773 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 09:03:44.853773 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 09:03:44.853773 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:03:44.853773 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:03:44.853773 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:03:44.853773 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:03:44.853773 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:03:44.853773 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:03:44.853773 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:03:44.853773 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 14 09:03:44.853773 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:03:44.853773 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:03:44.853773 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:03:44.853773 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:03:44.854723 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:03:44.854723 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:03:44.854723 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:03:44.854723 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:03:44.854723 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 09:03:44.854723 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 09:03:44.854723 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 09:03:44.854723 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 09:03:44.854723 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 09:03:44.854723 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 09:03:44.854723 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 09:03:44.854723 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 14 09:03:44.856928 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 14/29] 10.222.0.26 () {56 vars in 1096 bytes} [Thu Jul 14 09:03:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 392 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 09:03:44.858496 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e277aed1-4e2e-4789-88b8-27734cd6713d None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:03:44.858960 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e277aed1-4e2e-4789-88b8-27734cd6713d None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:03:44.859434 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e277aed1-4e2e-4789-88b8-27734cd6713d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:03:44.878552 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-e277aed1-4e2e-4789-88b8-27734cd6713d None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:03:44.880157 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-e277aed1-4e2e-4789-88b8-27734cd6713d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:03:44.888194 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:03:44.920892 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:03:44.942870 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b3b6c9-1582-47c4-b3a2-407b365e29a2 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:03:44.973396 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 15/30] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:03:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 09:03:44.975179 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dc2cab-273f-45b3-b095-ef2dd07980cf None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:03:44.975853 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dc2cab-273f-45b3-b095-ef2dd07980cf None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:03:44.976231 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dc2cab-273f-45b3-b095-ef2dd07980cf None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:03:44.978498 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 16/31] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:03:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:03:44.982030 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35b1fa7-4add-42ed-be7a-42f6727ffc5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:03:44.982861 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35b1fa7-4add-42ed-be7a-42f6727ffc5d None admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:03:44.983348 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35b1fa7-4add-42ed-be7a-42f6727ffc5d None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:03:44.983469 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69dc2cab-273f-45b3-b095-ef2dd07980cf None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:03:44.984003 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35b1fa7-4add-42ed-be7a-42f6727ffc5d None admin] PATH_INFO: `/v3/projects` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:03:44.984976 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35b1fa7-4add-42ed-be7a-42f6727ffc5d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:03:44.986937 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35b1fa7-4add-42ed-be7a-42f6727ffc5d None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:03:45.093036 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69dc2cab-273f-45b3-b095-ef2dd07980cf None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:03:45.128215 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8fe58e-2d0f-47a7-8a9b-dbd03854b3c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:03:45.128891 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8fe58e-2d0f-47a7-8a9b-dbd03854b3c6 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:03:45.129179 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8fe58e-2d0f-47a7-8a9b-dbd03854b3c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:03:45.129457 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8fe58e-2d0f-47a7-8a9b-dbd03854b3c6 None admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:03:45.130225 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8fe58e-2d0f-47a7-8a9b-dbd03854b3c6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:03:45.130534 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8fe58e-2d0f-47a7-8a9b-dbd03854b3c6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:03:45.131966 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8fe58e-2d0f-47a7-8a9b-dbd03854b3c6 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:03:45.155428 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 17/33] 10.222.0.26 () {56 vars in 1099 bytes} [Thu Jul 14 09:03:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:03:45.155428 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3d630f-5ca1-4bfd-ad28-2f609f31483d None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:03:45.155428 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3d630f-5ca1-4bfd-ad28-2f609f31483d None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:03:45.155428 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3d630f-5ca1-4bfd-ad28-2f609f31483d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:03:45.156043 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 16/33] 10.222.0.26 () {60 vars in 1141 bytes} [Thu Jul 14 09:03:44 2022] POST /identity/v3/projects => generated 301 bytes in 354 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 09:03:45.158956 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35b1fa7-4add-42ed-be7a-42f6727ffc5d None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:03:45.192000 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-0b3d630f-5ca1-4bfd-ad28-2f609f31483d None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:03:45.194195 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-0b3d630f-5ca1-4bfd-ad28-2f609f31483d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:03:45.253098 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:03:45.307217 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 18/34] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:03:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 09:03:45.307217 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3d630f-5ca1-4bfd-ad28-2f609f31483d None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:03:45.347861 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35b1fa7-4add-42ed-be7a-42f6727ffc5d None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:03:45.358840 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:03:45.402505 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76e93d0-9cd1-4f6d-8744-7677c4425e3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:03:45.403646 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76e93d0-9cd1-4f6d-8744-7677c4425e3a None admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:03:45.403940 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76e93d0-9cd1-4f6d-8744-7677c4425e3a None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:03:45.404223 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76e93d0-9cd1-4f6d-8744-7677c4425e3a None admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:03:45.404995 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76e93d0-9cd1-4f6d-8744-7677c4425e3a None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:03:45.406620 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76e93d0-9cd1-4f6d-8744-7677c4425e3a None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:03:45.424696 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 17/35] 10.222.0.26 () {60 vars in 1131 bytes} [Thu Jul 14 09:03:45 2022] POST /identity/v3/roles => generated 225 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 09:03:45.426097 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe10e69-f421-4335-8a18-5e5c52bae650 None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:03:45.426390 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe10e69-f421-4335-8a18-5e5c52bae650 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:03:45.426773 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe10e69-f421-4335-8a18-5e5c52bae650 None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:03:45.428519 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 18/36] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:03:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:03:45.429229 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe10e69-f421-4335-8a18-5e5c52bae650 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:03:45.442313 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3d630f-5ca1-4bfd-ad28-2f609f31483d None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:03:45.486632 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2830856-c006-44a0-9392-6545eba64a65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:03:45.487761 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2830856-c006-44a0-9392-6545eba64a65 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:03:45.488116 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2830856-c006-44a0-9392-6545eba64a65 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:03:45.488460 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2830856-c006-44a0-9392-6545eba64a65 None admin] PATH_INFO: `/v3/users/d00c29cc14a043099fbeda47a3389faa` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:03:45.490472 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2830856-c006-44a0-9392-6545eba64a65 None admin] RBAC: Authorizing `identity:get_user(user_id=d00c29cc14a043099fbeda47a3389faa)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:03:45.492315 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2830856-c006-44a0-9392-6545eba64a65 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:03:45.494354 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 19/37] 10.222.0.26 () {56 vars in 1177 bytes} [Thu Jul 14 09:03:45 2022] GET /identity/v3/users/d00c29cc14a043099fbeda47a3389faa => generated 247 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:03:45.495850 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87539004-c394-4097-baa4-0dbfb73e132d None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:03:45.496208 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87539004-c394-4097-baa4-0dbfb73e132d None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:03:45.496592 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87539004-c394-4097-baa4-0dbfb73e132d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:03:45.519367 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-87539004-c394-4097-baa4-0dbfb73e132d None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:03:45.521099 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-87539004-c394-4097-baa4-0dbfb73e132d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:03:45.547321 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:03:45.601874 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 20/38] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:03:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 09:03:45.602648 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87539004-c394-4097-baa4-0dbfb73e132d None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:03:45.668792 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe10e69-f421-4335-8a18-5e5c52bae650 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:03:45.703532 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdeda75d-1bdc-4fd2-bdba-c5ce6a2c670c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:03:45.704210 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeda75d-1bdc-4fd2-bdba-c5ce6a2c670c None admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:03:45.704495 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeda75d-1bdc-4fd2-bdba-c5ce6a2c670c None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:03:45.704770 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeda75d-1bdc-4fd2-bdba-c5ce6a2c670c None admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:03:45.705530 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdeda75d-1bdc-4fd2-bdba-c5ce6a2c670c None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:03:45.706927 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdeda75d-1bdc-4fd2-bdba-c5ce6a2c670c None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:03:45.711356 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87539004-c394-4097-baa4-0dbfb73e132d None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:03:45.720064 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 19/39] 10.222.0.26 () {60 vars in 1131 bytes} [Thu Jul 14 09:03:45 2022] POST /identity/v3/roles => generated 219 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 09:03:45.721309 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69578a6a-30f1-4299-bb99-f5e8662aa450 None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:03:45.721592 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69578a6a-30f1-4299-bb99-f5e8662aa450 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:03:45.721863 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69578a6a-30f1-4299-bb99-f5e8662aa450 None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:03:45.723295 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 20/40] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:03:45 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:03:45.728031 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69578a6a-30f1-4299-bb99-f5e8662aa450 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:03:45.750400 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddd43a7-eb6b-4983-a4c5-1bc40701d389 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:03:45.751243 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddd43a7-eb6b-4983-a4c5-1bc40701d389 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:03:45.751509 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddd43a7-eb6b-4983-a4c5-1bc40701d389 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:03:45.751783 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddd43a7-eb6b-4983-a4c5-1bc40701d389 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:03:45.753190 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddd43a7-eb6b-4983-a4c5-1bc40701d389 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:03:45.754734 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddd43a7-eb6b-4983-a4c5-1bc40701d389 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:03:45.761488 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 21/41] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jul 14 09:03:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:03:45.761488 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c04c63-9cac-4279-a8e9-5a9c9a01a2f3 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:03:45.761488 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c04c63-9cac-4279-a8e9-5a9c9a01a2f3 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:03:45.761488 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c04c63-9cac-4279-a8e9-5a9c9a01a2f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:03:45.771354 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:03:45.777482 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-d9c04c63-9cac-4279-a8e9-5a9c9a01a2f3 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:03:45.778703 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-d9c04c63-9cac-4279-a8e9-5a9c9a01a2f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:03:45.797778 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:03:45.856685 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 22/42] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:03:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 09:03:45.857529 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c04c63-9cac-4279-a8e9-5a9c9a01a2f3 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:03:45.887015 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69578a6a-30f1-4299-bb99-f5e8662aa450 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:03:45.927157 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28f8701-f4f4-41a9-bc0a-8d2f032bf19a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:03:45.928040 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28f8701-f4f4-41a9-bc0a-8d2f032bf19a None admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:03:45.928231 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28f8701-f4f4-41a9-bc0a-8d2f032bf19a None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:03:45.928475 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28f8701-f4f4-41a9-bc0a-8d2f032bf19a None admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:03:45.929235 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28f8701-f4f4-41a9-bc0a-8d2f032bf19a None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:03:45.930785 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28f8701-f4f4-41a9-bc0a-8d2f032bf19a None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:03:45.946218 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 21/43] 10.222.0.26 () {60 vars in 1131 bytes} [Thu Jul 14 09:03:45 2022] POST /identity/v3/roles => generated 223 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 09:03:45.947049 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28f8701-f4f4-41a9-bc0a-8d2f032bf19a None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:03:45.994107 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c04c63-9cac-4279-a8e9-5a9c9a01a2f3 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:03:46.030841 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a5e7be-999a-437e-b274-c76e5a04c94f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:03:46.031772 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a5e7be-999a-437e-b274-c76e5a04c94f None admin] REQUEST_METHOD: `PUT` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:03:46.032064 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a5e7be-999a-437e-b274-c76e5a04c94f None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:03:46.032327 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a5e7be-999a-437e-b274-c76e5a04c94f None admin] PATH_INFO: `/v3/domains/default/users/d00c29cc14a043099fbeda47a3389faa/roles/7b549d58d1724395b343609d895c80d0` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:03:46.034471 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a5e7be-999a-437e-b274-c76e5a04c94f None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=d00c29cc14a043099fbeda47a3389faa, role_id=7b549d58d1724395b343609d895c80d0)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:03:46.035993 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a5e7be-999a-437e-b274-c76e5a04c94f None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:03:46.043254 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:03:46.058138 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28f8701-f4f4-41a9-bc0a-8d2f032bf19a None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:03:46.095845 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ce9f4d-f52c-48b2-b815-66395e1b19b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:03:46.096663 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ce9f4d-f52c-48b2-b815-66395e1b19b5 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:03:46.096952 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ce9f4d-f52c-48b2-b815-66395e1b19b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:03:46.097232 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ce9f4d-f52c-48b2-b815-66395e1b19b5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:03:46.098757 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ce9f4d-f52c-48b2-b815-66395e1b19b5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:03:46.100264 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ce9f4d-f52c-48b2-b815-66395e1b19b5 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:03:46.102105 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 22/44] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jul 14 09:03:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:03:46.107459 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ce9f4d-f52c-48b2-b815-66395e1b19b5 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:03:46.155978 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 23/45] 10.222.0.26 () {58 vars in 1361 bytes} [Thu Jul 14 09:03:45 2022] PUT /identity/v3/domains/default/users/d00c29cc14a043099fbeda47a3389faa/roles/7b549d58d1724395b343609d895c80d0 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:03:46.219999 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ce9f4d-f52c-48b2-b815-66395e1b19b5 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:03:46.259126 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc18e5e-cf0a-4f95-a934-84d7f171cd79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:03:46.259838 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc18e5e-cf0a-4f95-a934-84d7f171cd79 None admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:03:46.260126 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc18e5e-cf0a-4f95-a934-84d7f171cd79 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:03:46.260402 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc18e5e-cf0a-4f95-a934-84d7f171cd79 None admin] PATH_INFO: `/v3/projects` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:03:46.261143 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc18e5e-cf0a-4f95-a934-84d7f171cd79 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:03:46.262623 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc18e5e-cf0a-4f95-a934-84d7f171cd79 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:03:46.291538 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 23/46] 10.222.0.26 () {60 vars in 1141 bytes} [Thu Jul 14 09:03:46 2022] POST /identity/v3/projects => generated 312 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 09:03:48.152832 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d472825-d03e-47ee-a1b8-fe0a5c080ce0 None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:03:48.153372 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d472825-d03e-47ee-a1b8-fe0a5c080ce0 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:03:48.153455 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d472825-d03e-47ee-a1b8-fe0a5c080ce0 None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:03:48.159629 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 24/47] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:03:48 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:03:48.166463 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20824f9a-618c-40a9-9545-7981bb9505af None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:03:48.167058 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20824f9a-618c-40a9-9545-7981bb9505af None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:03:48.167645 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20824f9a-618c-40a9-9545-7981bb9505af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:03:48.189348 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-20824f9a-618c-40a9-9545-7981bb9505af None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:03:48.190887 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-20824f9a-618c-40a9-9545-7981bb9505af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:03:48.192972 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:03:48.262072 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 24/48] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:03:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 09:03:48.284795 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d472825-d03e-47ee-a1b8-fe0a5c080ce0 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:03:48.290781 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c34635-c039-4b9b-99e3-f249b10a5060 None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:03:48.291177 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c34635-c039-4b9b-99e3-f249b10a5060 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:03:48.291659 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c34635-c039-4b9b-99e3-f249b10a5060 None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:03:48.293941 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 25/49] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:03:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:03:48.299972 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b809192b-30b1-41b7-a12d-a2d4dd22b2f8 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:03:48.300359 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b809192b-30b1-41b7-a12d-a2d4dd22b2f8 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:03:48.300788 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b809192b-30b1-41b7-a12d-a2d4dd22b2f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:03:48.322965 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-b809192b-30b1-41b7-a12d-a2d4dd22b2f8 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:03:48.324244 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-b809192b-30b1-41b7-a12d-a2d4dd22b2f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:03:48.398571 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 26/50] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:03:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 09:03:48.412120 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:03:48.422020 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b809192b-30b1-41b7-a12d-a2d4dd22b2f8 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:03:48.505622 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:03:48.518716 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d472825-d03e-47ee-a1b8-fe0a5c080ce0 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:03:48.543367 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b809192b-30b1-41b7-a12d-a2d4dd22b2f8 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:03:48.555941 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41f282a-5bbe-43d7-ab53-140bf63337f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:03:48.556744 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41f282a-5bbe-43d7-ab53-140bf63337f1 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:03:48.556994 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41f282a-5bbe-43d7-ab53-140bf63337f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:03:48.557299 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41f282a-5bbe-43d7-ab53-140bf63337f1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:03:48.567852 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41f282a-5bbe-43d7-ab53-140bf63337f1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 09:03:48.575508 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-e41f282a-5bbe-43d7-ab53-140bf63337f1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 09:03:48.575508 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:03:48.575508 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:03:48.575508 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:03:48.575508 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:03:48.575508 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:03:48.575508 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:03:48.575508 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:03:48.575508 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:03:48.575508 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:03:48.575508 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:03:48.575508 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:03:48.575508 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 09:03:48.575508 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 09:03:48.575508 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 09:03:48.575508 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 09:03:48.576523 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 09:03:48.576523 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 09:03:48.576523 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 09:03:48.576523 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 09:03:48.576523 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:03:48.576523 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:03:48.576523 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:03:48.576523 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:03:48.576523 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:03:48.576523 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:03:48.576523 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:03:48.576523 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 14 09:03:48.576523 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:03:48.576523 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:03:48.576523 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:03:48.576523 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:03:48.577569 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:03:48.577569 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:03:48.577569 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:03:48.577569 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:03:48.577569 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 09:03:48.577569 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 09:03:48.577569 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 09:03:48.577569 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 09:03:48.577569 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 09:03:48.577569 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 09:03:48.577569 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 09:03:48.577569 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 14 09:03:48.577569 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 25/51] 10.222.0.26 () {56 vars in 1096 bytes} [Thu Jul 14 09:03:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:03:48.582331 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd80e1b-3017-40ee-a05d-152c7fd3a293 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:03:48.582839 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41f282a-5bbe-43d7-ab53-140bf63337f1 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:03:48.583136 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd80e1b-3017-40ee-a05d-152c7fd3a293 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:03:48.583453 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd80e1b-3017-40ee-a05d-152c7fd3a293 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:03:48.583724 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd80e1b-3017-40ee-a05d-152c7fd3a293 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:03:48.584892 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd80e1b-3017-40ee-a05d-152c7fd3a293 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:03:48.586365 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd80e1b-3017-40ee-a05d-152c7fd3a293 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:03:48.588050 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 27/52] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jul 14 09:03:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:03:48.593373 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd80e1b-3017-40ee-a05d-152c7fd3a293 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:03:48.657419 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:03:48.682497 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41f282a-5bbe-43d7-ab53-140bf63337f1 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:03:48.692234 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd80e1b-3017-40ee-a05d-152c7fd3a293 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:03:48.715827 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a582586-634d-4a00-b6d6-3aa7a150dec3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:03:48.716495 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a582586-634d-4a00-b6d6-3aa7a150dec3 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:03:48.716787 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a582586-634d-4a00-b6d6-3aa7a150dec3 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:03:48.717062 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a582586-634d-4a00-b6d6-3aa7a150dec3 None admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:03:48.717823 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a582586-634d-4a00-b6d6-3aa7a150dec3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:03:48.718173 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a582586-634d-4a00-b6d6-3aa7a150dec3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:03:48.719772 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a582586-634d-4a00-b6d6-3aa7a150dec3 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:03:48.725534 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ba3b74-ddcf-4f81-aae2-702dba294db8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:03:48.726201 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ba3b74-ddcf-4f81-aae2-702dba294db8 None admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:03:48.726484 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ba3b74-ddcf-4f81-aae2-702dba294db8 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:03:48.726761 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ba3b74-ddcf-4f81-aae2-702dba294db8 None admin] PATH_INFO: `/v3/projects` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:03:48.727518 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ba3b74-ddcf-4f81-aae2-702dba294db8 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:03:48.728238 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 26/53] 10.222.0.26 () {56 vars in 1099 bytes} [Thu Jul 14 09:03:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:03:48.728949 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ba3b74-ddcf-4f81-aae2-702dba294db8 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:03:48.733418 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:03:48.733805 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a582586-634d-4a00-b6d6-3aa7a150dec3 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:03:48.755832 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 28/54] 10.222.0.26 () {60 vars in 1140 bytes} [Thu Jul 14 09:03:48 2022] POST /identity/v3/projects => generated 298 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 09:03:48.849748 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a582586-634d-4a00-b6d6-3aa7a150dec3 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:03:48.887394 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038302b5-74c5-4cf8-830d-8f99afac1f83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:03:48.888296 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038302b5-74c5-4cf8-830d-8f99afac1f83 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:03:48.888585 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038302b5-74c5-4cf8-830d-8f99afac1f83 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:03:48.888864 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038302b5-74c5-4cf8-830d-8f99afac1f83 None admin] PATH_INFO: `/v3/users/d00c29cc14a043099fbeda47a3389faa` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:03:48.890307 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038302b5-74c5-4cf8-830d-8f99afac1f83 None admin] RBAC: Authorizing `identity:get_user(user_id=d00c29cc14a043099fbeda47a3389faa)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:03:48.891825 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038302b5-74c5-4cf8-830d-8f99afac1f83 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:03:48.893516 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 27/55] 10.222.0.26 () {56 vars in 1177 bytes} [Thu Jul 14 09:03:48 2022] GET /identity/v3/users/d00c29cc14a043099fbeda47a3389faa => generated 247 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 09:03:48.899170 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ba3b74-ddcf-4f81-aae2-702dba294db8 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:03:49.003609 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ba3b74-ddcf-4f81-aae2-702dba294db8 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:03:49.037843 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a582de8a-729c-42f6-9be9-2fe402c98147 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:03:49.038637 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a582de8a-729c-42f6-9be9-2fe402c98147 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:03:49.038916 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a582de8a-729c-42f6-9be9-2fe402c98147 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:03:49.039201 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a582de8a-729c-42f6-9be9-2fe402c98147 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:03:49.040385 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a582de8a-729c-42f6-9be9-2fe402c98147 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:03:49.041837 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a582de8a-729c-42f6-9be9-2fe402c98147 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:03:49.043516 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 29/56] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jul 14 09:03:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:03:49.049400 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038302b5-74c5-4cf8-830d-8f99afac1f83 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:03:49.053388 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:03:49.150212 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038302b5-74c5-4cf8-830d-8f99afac1f83 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:03:49.184387 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5458e2fe-db83-47d8-b551-de6c43998f3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:03:49.184996 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5458e2fe-db83-47d8-b551-de6c43998f3e None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:03:49.185317 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5458e2fe-db83-47d8-b551-de6c43998f3e None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:03:49.185592 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5458e2fe-db83-47d8-b551-de6c43998f3e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:03:49.186481 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5458e2fe-db83-47d8-b551-de6c43998f3e None admin] RBAC: Adding query filter params (role.id=7b549d58d1724395b343609d895c80d0, scope.domain.id=default, user.id=d00c29cc14a043099fbeda47a3389faa) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:03:49.186790 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5458e2fe-db83-47d8-b551-de6c43998f3e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:03:49.188271 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5458e2fe-db83-47d8-b551-de6c43998f3e None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:03:49.196500 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 28/57] 10.222.0.26 () {56 vars in 1322 bytes} [Thu Jul 14 09:03:49 2022] GET /identity/v3/role_assignments?user.id=d00c29cc14a043099fbeda47a3389faa&scope.domain.id=default&role.id=7b549d58d1724395b343609d895c80d0 => generated 530 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:03:50.776149 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcef312-6103-4358-a9ae-f1fc7f7edcaa None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:03:50.776675 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcef312-6103-4358-a9ae-f1fc7f7edcaa None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:03:50.776760 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcef312-6103-4358-a9ae-f1fc7f7edcaa None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:03:50.778834 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 30/58] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:03:50 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:03:50.787405 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45eb172-b301-420d-b81d-5f422aa6debb None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:03:50.787794 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45eb172-b301-420d-b81d-5f422aa6debb None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:03:50.788058 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45eb172-b301-420d-b81d-5f422aa6debb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:03:50.809412 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-d45eb172-b301-420d-b81d-5f422aa6debb None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:03:50.810915 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-d45eb172-b301-420d-b81d-5f422aa6debb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:03:50.885700 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:03:50.892505 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 29/59] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:03:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 09:03:50.927882 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcef312-6103-4358-a9ae-f1fc7f7edcaa None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:03:51.001668 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:03:51.052117 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcef312-6103-4358-a9ae-f1fc7f7edcaa None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:03:51.089613 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb5b35e-e993-44ca-8884-ffb751cb1e1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:03:51.090342 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb5b35e-e993-44ca-8884-ffb751cb1e1b None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:03:51.090631 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb5b35e-e993-44ca-8884-ffb751cb1e1b None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:03:51.090910 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb5b35e-e993-44ca-8884-ffb751cb1e1b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:03:51.092258 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb5b35e-e993-44ca-8884-ffb751cb1e1b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:03:51.093702 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb5b35e-e993-44ca-8884-ffb751cb1e1b None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:03:51.095345 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 31/60] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jul 14 09:03:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:03:51.103363 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45eb172-b301-420d-b81d-5f422aa6debb None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:03:51.209471 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45eb172-b301-420d-b81d-5f422aa6debb None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:03:51.247743 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e542d09-f726-47cd-ba1a-30fdca3bf397 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:03:51.248554 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e542d09-f726-47cd-ba1a-30fdca3bf397 None admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:03:51.248902 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e542d09-f726-47cd-ba1a-30fdca3bf397 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:03:51.249276 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e542d09-f726-47cd-ba1a-30fdca3bf397 None admin] PATH_INFO: `/v3/users` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:03:51.249937 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e542d09-f726-47cd-ba1a-30fdca3bf397 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:03:51.251424 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e542d09-f726-47cd-ba1a-30fdca3bf397 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:03:51.280437 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 30/61] 10.222.0.26 () {60 vars in 1132 bytes} [Thu Jul 14 09:03:51 2022] POST /identity/v3/users => generated 275 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 09:03:54.345264 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417b8c3f-ed32-4a53-8330-283ac0f3512b None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:03:54.346121 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417b8c3f-ed32-4a53-8330-283ac0f3512b None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:03:54.346563 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417b8c3f-ed32-4a53-8330-283ac0f3512b None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:03:54.348127 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 32/62] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:03:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:03:54.353540 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717cf1d5-adf1-430c-b8da-10f8c2f284a1 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:03:54.354259 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717cf1d5-adf1-430c-b8da-10f8c2f284a1 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:03:54.354673 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717cf1d5-adf1-430c-b8da-10f8c2f284a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:03:54.380184 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-717cf1d5-adf1-430c-b8da-10f8c2f284a1 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:03:54.381603 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-717cf1d5-adf1-430c-b8da-10f8c2f284a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:03:54.450925 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:03:54.473967 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 31/63] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:03:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 09:03:54.514632 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717cf1d5-adf1-430c-b8da-10f8c2f284a1 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:03:54.515563 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ec06fa-7004-4db4-bc7b-6e3829f2cc31 None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:03:54.516051 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ec06fa-7004-4db4-bc7b-6e3829f2cc31 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:03:54.516520 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ec06fa-7004-4db4-bc7b-6e3829f2cc31 None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:03:54.518108 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 33/64] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:03:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:03:54.523383 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5540c44e-10bd-4a04-8dae-c7d8f35697ce None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:03:54.523918 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5540c44e-10bd-4a04-8dae-c7d8f35697ce None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:03:54.524397 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5540c44e-10bd-4a04-8dae-c7d8f35697ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:03:54.555878 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-5540c44e-10bd-4a04-8dae-c7d8f35697ce None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:03:54.557407 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-5540c44e-10bd-4a04-8dae-c7d8f35697ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:03:54.622274 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717cf1d5-adf1-430c-b8da-10f8c2f284a1 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:03:54.630919 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 34/65] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:03:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 09:03:54.674581 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d164d0-5f6f-4841-a4fa-09e01fbb1648 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:03:54.675418 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d164d0-5f6f-4841-a4fa-09e01fbb1648 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:03:54.675705 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d164d0-5f6f-4841-a4fa-09e01fbb1648 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:03:54.675983 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d164d0-5f6f-4841-a4fa-09e01fbb1648 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:03:54.692536 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d164d0-5f6f-4841-a4fa-09e01fbb1648 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 09:03:54.696294 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5540c44e-10bd-4a04-8dae-c7d8f35697ce None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:03:54.700429 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-90d164d0-5f6f-4841-a4fa-09e01fbb1648 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 09:03:54.700429 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:03:54.700429 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:03:54.700429 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:03:54.700429 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:03:54.700429 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:03:54.700429 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:03:54.700429 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:03:54.700429 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:03:54.700429 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:03:54.700429 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:03:54.700429 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:03:54.700429 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 09:03:54.700429 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 09:03:54.700429 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 09:03:54.700429 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 09:03:54.701287 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 09:03:54.701287 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 09:03:54.701287 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 09:03:54.701287 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 09:03:54.701287 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:03:54.701287 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:03:54.701287 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:03:54.701287 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:03:54.701287 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:03:54.701287 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:03:54.701287 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:03:54.701287 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 14 09:03:54.701287 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:03:54.701287 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:03:54.701287 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:03:54.701287 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:03:54.702127 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:03:54.702127 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:03:54.702127 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:03:54.702127 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:03:54.702127 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 09:03:54.702127 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 09:03:54.702127 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 09:03:54.702127 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 09:03:54.702127 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 09:03:54.702127 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 09:03:54.702127 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 09:03:54.702127 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 14 09:03:54.702127 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 32/66] 10.222.0.26 () {56 vars in 1099 bytes} [Thu Jul 14 09:03:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 09:03:54.707558 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d164d0-5f6f-4841-a4fa-09e01fbb1648 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:03:54.764769 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:03:54.766980 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:03:54.812707 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5540c44e-10bd-4a04-8dae-c7d8f35697ce None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:03:54.825358 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d164d0-5f6f-4841-a4fa-09e01fbb1648 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:03:54.849163 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6454328-f83d-42f9-aab4-0a5a0c3e51e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:03:54.849969 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6454328-f83d-42f9-aab4-0a5a0c3e51e7 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:03:54.850234 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6454328-f83d-42f9-aab4-0a5a0c3e51e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:03:54.850515 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6454328-f83d-42f9-aab4-0a5a0c3e51e7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:03:54.861119 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6454328-f83d-42f9-aab4-0a5a0c3e51e7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 09:03:54.862894 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9507e00c-294d-451c-95ce-ab08c786f141 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:03:54.863782 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9507e00c-294d-451c-95ce-ab08c786f141 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:03:54.864256 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9507e00c-294d-451c-95ce-ab08c786f141 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:03:54.864694 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9507e00c-294d-451c-95ce-ab08c786f141 None admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:03:54.865612 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9507e00c-294d-451c-95ce-ab08c786f141 None admin] RBAC: Adding query filter params (name=member) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:03:54.866082 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9507e00c-294d-451c-95ce-ab08c786f141 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:03:54.867683 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9507e00c-294d-451c-95ce-ab08c786f141 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:03:54.868887 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-d6454328-f83d-42f9-aab4-0a5a0c3e51e7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 09:03:54.868887 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:03:54.868887 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:03:54.868887 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:03:54.868887 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:03:54.868887 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:03:54.868887 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:03:54.868887 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:03:54.868887 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:03:54.868887 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:03:54.868887 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:03:54.868887 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:03:54.868887 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 09:03:54.868887 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 09:03:54.868887 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 09:03:54.868887 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 09:03:54.869705 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 09:03:54.869705 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 09:03:54.869705 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 09:03:54.869705 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 09:03:54.869705 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:03:54.869705 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:03:54.869705 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:03:54.869705 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:03:54.869705 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:03:54.869705 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:03:54.869705 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:03:54.869705 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 14 09:03:54.869705 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:03:54.869705 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:03:54.869705 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:03:54.869705 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:03:54.870482 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:03:54.870482 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:03:54.870482 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:03:54.870482 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:03:54.870482 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 09:03:54.870482 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 09:03:54.870482 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 09:03:54.870482 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 09:03:54.870482 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 09:03:54.870482 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 09:03:54.870482 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 09:03:54.870482 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 14 09:03:54.870482 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 35/67] 10.222.0.26 () {56 vars in 1096 bytes} [Thu Jul 14 09:03:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 09:03:54.871463 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e977d22f-9dfb-4b8e-a885-394b11b5b7ad None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:03:54.871800 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e977d22f-9dfb-4b8e-a885-394b11b5b7ad None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:03:54.872165 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e977d22f-9dfb-4b8e-a885-394b11b5b7ad None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:03:54.875601 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 36/68] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:03:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:03:54.875601 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f327d5-e4f6-48ba-bbf4-cec443347671 None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:03:54.875601 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f327d5-e4f6-48ba-bbf4-cec443347671 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:03:54.875828 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f327d5-e4f6-48ba-bbf4-cec443347671 None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:03:54.877041 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 37/69] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:03:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:03:54.878638 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f327d5-e4f6-48ba-bbf4-cec443347671 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:03:54.885859 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 33/70] 10.222.0.26 () {56 vars in 1101 bytes} [Thu Jul 14 09:03:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 09:03:54.887127 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cb77b2-b9b8-4cee-b612-d625d478f400 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:03:54.887445 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cb77b2-b9b8-4cee-b612-d625d478f400 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:03:54.887788 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cb77b2-b9b8-4cee-b612-d625d478f400 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:03:54.907944 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-f0cb77b2-b9b8-4cee-b612-d625d478f400 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:03:54.909285 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-f0cb77b2-b9b8-4cee-b612-d625d478f400 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:03:54.987276 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f327d5-e4f6-48ba-bbf4-cec443347671 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:03:55.001153 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 34/71] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:03:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 09:03:55.001153 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea749e8-d18d-4fc7-9f2d-72fafd872108 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:03:55.001153 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea749e8-d18d-4fc7-9f2d-72fafd872108 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:03:55.001153 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea749e8-d18d-4fc7-9f2d-72fafd872108 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:03:55.003327 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:03:55.025183 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-9ea749e8-d18d-4fc7-9f2d-72fafd872108 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:03:55.026380 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-9ea749e8-d18d-4fc7-9f2d-72fafd872108 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:03:55.031304 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6a43e5-d33b-4ef3-81b3-7c2da7d0c561 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:03:55.032115 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6a43e5-d33b-4ef3-81b3-7c2da7d0c561 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:03:55.032588 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6a43e5-d33b-4ef3-81b3-7c2da7d0c561 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:03:55.033011 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6a43e5-d33b-4ef3-81b3-7c2da7d0c561 None admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:03:55.033957 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6a43e5-d33b-4ef3-81b3-7c2da7d0c561 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:03:55.034384 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6a43e5-d33b-4ef3-81b3-7c2da7d0c561 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:03:55.036031 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6a43e5-d33b-4ef3-81b3-7c2da7d0c561 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:03:55.046129 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 38/72] 10.222.0.26 () {56 vars in 1099 bytes} [Thu Jul 14 09:03:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:03:55.047060 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6a43e5-d33b-4ef3-81b3-7c2da7d0c561 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:03:55.065202 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:03:55.113358 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 35/73] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:03:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 09:03:55.114219 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea749e8-d18d-4fc7-9f2d-72fafd872108 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:03:55.160613 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6a43e5-d33b-4ef3-81b3-7c2da7d0c561 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:03:55.197016 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15238ab-3fbf-41b8-bc21-9d24fa92e4d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:03:55.197904 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15238ab-3fbf-41b8-bc21-9d24fa92e4d7 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:03:55.198188 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15238ab-3fbf-41b8-bc21-9d24fa92e4d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:03:55.198465 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15238ab-3fbf-41b8-bc21-9d24fa92e4d7 None admin] PATH_INFO: `/v3/users/cb553a2f9c234bcbadbe3b9dbdd4a92e` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:03:55.212158 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15238ab-3fbf-41b8-bc21-9d24fa92e4d7 None admin] RBAC: Authorizing `identity:get_user(user_id=cb553a2f9c234bcbadbe3b9dbdd4a92e)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:03:55.213628 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15238ab-3fbf-41b8-bc21-9d24fa92e4d7 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:03:55.215316 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 39/74] 10.222.0.26 () {56 vars in 1177 bytes} [Thu Jul 14 09:03:55 2022] GET /identity/v3/users/cb553a2f9c234bcbadbe3b9dbdd4a92e => generated 275 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:03:55.216149 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15238ab-3fbf-41b8-bc21-9d24fa92e4d7 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:03:55.228778 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea749e8-d18d-4fc7-9f2d-72fafd872108 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:03:55.252040 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:03:55.263998 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67580dc2-9988-486b-815a-eb621a154c06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:03:55.264811 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67580dc2-9988-486b-815a-eb621a154c06 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:03:55.265098 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67580dc2-9988-486b-815a-eb621a154c06 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:03:55.265378 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67580dc2-9988-486b-815a-eb621a154c06 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:03:55.275637 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67580dc2-9988-486b-815a-eb621a154c06 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 09:03:55.284072 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-67580dc2-9988-486b-815a-eb621a154c06 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 09:03:55.284072 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:03:55.284072 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:03:55.284072 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:03:55.284072 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:03:55.284072 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:03:55.284072 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:03:55.284072 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:03:55.284072 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:03:55.284072 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:03:55.284072 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:03:55.284072 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:03:55.284072 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 09:03:55.284072 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 09:03:55.284072 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 09:03:55.284072 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 09:03:55.284968 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 09:03:55.284968 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 09:03:55.284968 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 09:03:55.284968 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 09:03:55.284968 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:03:55.284968 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:03:55.284968 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:03:55.284968 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:03:55.284968 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:03:55.284968 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:03:55.284968 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:03:55.284968 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 14 09:03:55.284968 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:03:55.284968 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:03:55.284968 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:03:55.284968 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:03:55.285754 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:03:55.285754 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:03:55.285754 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:03:55.285754 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:03:55.285754 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 09:03:55.285754 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 09:03:55.285754 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 09:03:55.285754 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 09:03:55.285754 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 09:03:55.285754 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 09:03:55.285754 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 09:03:55.285754 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 14 09:03:55.285754 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 36/75] 10.222.0.26 () {56 vars in 1099 bytes} [Thu Jul 14 09:03:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:03:55.286407 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67580dc2-9988-486b-815a-eb621a154c06 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:03:55.318538 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:03:55.322462 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15238ab-3fbf-41b8-bc21-9d24fa92e4d7 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:03:55.356090 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adfdf3a-89fa-4ca6-90a2-530866684a41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:03:55.357028 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adfdf3a-89fa-4ca6-90a2-530866684a41 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:03:55.357321 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adfdf3a-89fa-4ca6-90a2-530866684a41 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:03:55.357602 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adfdf3a-89fa-4ca6-90a2-530866684a41 None admin] PATH_INFO: `/v3/users/d00c29cc14a043099fbeda47a3389faa` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:03:55.358826 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3adfdf3a-89fa-4ca6-90a2-530866684a41 None admin] RBAC: Authorizing `identity:get_user(user_id=d00c29cc14a043099fbeda47a3389faa)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:03:55.360498 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3adfdf3a-89fa-4ca6-90a2-530866684a41 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:03:55.362223 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 40/76] 10.222.0.26 () {56 vars in 1177 bytes} [Thu Jul 14 09:03:55 2022] GET /identity/v3/users/d00c29cc14a043099fbeda47a3389faa => generated 247 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:03:55.365047 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46343ba-8bb2-4b17-89ab-49ee2bc41c56 None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:03:55.365047 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46343ba-8bb2-4b17-89ab-49ee2bc41c56 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:03:55.365047 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46343ba-8bb2-4b17-89ab-49ee2bc41c56 None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:03:55.365620 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 41/77] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:03:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:03:55.366274 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46343ba-8bb2-4b17-89ab-49ee2bc41c56 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:03:55.394538 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67580dc2-9988-486b-815a-eb621a154c06 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:03:55.435609 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472c073d-1a29-4f60-949d-b0c38d924901 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:03:55.436455 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472c073d-1a29-4f60-949d-b0c38d924901 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:03:55.436941 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472c073d-1a29-4f60-949d-b0c38d924901 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:03:55.437066 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472c073d-1a29-4f60-949d-b0c38d924901 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:03:55.448120 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472c073d-1a29-4f60-949d-b0c38d924901 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 09:03:55.456295 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-472c073d-1a29-4f60-949d-b0c38d924901 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 09:03:55.456295 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:03:55.456295 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:03:55.456295 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:03:55.456295 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:03:55.456295 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:03:55.456295 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:03:55.456295 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:03:55.456295 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:03:55.456295 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:03:55.456295 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:03:55.456295 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:03:55.456295 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 09:03:55.456295 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 09:03:55.456295 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 09:03:55.456295 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 09:03:55.457166 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 09:03:55.457166 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 09:03:55.457166 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 09:03:55.457166 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 09:03:55.457166 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:03:55.457166 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:03:55.457166 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:03:55.457166 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:03:55.457166 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:03:55.457166 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:03:55.457166 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:03:55.457166 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 14 09:03:55.457166 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:03:55.457166 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:03:55.457166 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:03:55.457166 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:03:55.457915 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:03:55.457915 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:03:55.457915 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:03:55.457915 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:03:55.457915 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 09:03:55.457915 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 09:03:55.457915 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 09:03:55.457915 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 09:03:55.457915 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 09:03:55.457915 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 09:03:55.457915 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 09:03:55.457915 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 14 09:03:55.457915 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 37/78] 10.222.0.26 () {56 vars in 1114 bytes} [Thu Jul 14 09:03:55 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:03:55.458620 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472c073d-1a29-4f60-949d-b0c38d924901 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:03:55.489753 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46343ba-8bb2-4b17-89ab-49ee2bc41c56 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:03:55.491351 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:03:55.524114 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fe05dc-5aba-4504-a65a-b34639f9b4fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:03:55.524777 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fe05dc-5aba-4504-a65a-b34639f9b4fb None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:03:55.525063 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fe05dc-5aba-4504-a65a-b34639f9b4fb None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:03:55.525441 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fe05dc-5aba-4504-a65a-b34639f9b4fb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:03:55.559637 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:03:55.579929 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472c073d-1a29-4f60-949d-b0c38d924901 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:03:55.599821 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fe05dc-5aba-4504-a65a-b34639f9b4fb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:03:55.601303 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fe05dc-5aba-4504-a65a-b34639f9b4fb None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:03:55.617468 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7034d4-b514-4e91-a070-624d5fc33240 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:03:55.618387 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7034d4-b514-4e91-a070-624d5fc33240 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:03:55.618931 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7034d4-b514-4e91-a070-624d5fc33240 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:03:55.619556 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7034d4-b514-4e91-a070-624d5fc33240 None admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:03:55.620323 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7034d4-b514-4e91-a070-624d5fc33240 None admin] RBAC: Adding query filter params (name=member) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:03:55.620635 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7034d4-b514-4e91-a070-624d5fc33240 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:03:55.622074 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7034d4-b514-4e91-a070-624d5fc33240 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:03:55.630476 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 38/79] 10.222.0.26 () {56 vars in 1101 bytes} [Thu Jul 14 09:03:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:03:55.631310 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7034d4-b514-4e91-a070-624d5fc33240 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:03:55.711667 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 42/80] 10.222.0.26 () {58 vars in 1282 bytes} [Thu Jul 14 09:03:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 09:03:55.712954 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c2c2c3-b45e-4995-8b11-24e02a92c37a None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:03:55.713249 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c2c2c3-b45e-4995-8b11-24e02a92c37a None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:03:55.713528 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c2c2c3-b45e-4995-8b11-24e02a92c37a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:03:55.718504 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:03:55.730891 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-68c2c2c3-b45e-4995-8b11-24e02a92c37a None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:03:55.732280 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-68c2c2c3-b45e-4995-8b11-24e02a92c37a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:03:55.734083 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7034d4-b514-4e91-a070-624d5fc33240 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:03:55.766927 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f68bce-dbf0-485b-83b1-dd42dcb18cb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:03:55.767621 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f68bce-dbf0-485b-83b1-dd42dcb18cb4 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:03:55.767908 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f68bce-dbf0-485b-83b1-dd42dcb18cb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:03:55.768207 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f68bce-dbf0-485b-83b1-dd42dcb18cb4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:03:55.785754 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:03:55.799461 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 43/81] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:03:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 09:03:55.800289 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c2c2c3-b45e-4995-8b11-24e02a92c37a None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:03:55.844663 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f68bce-dbf0-485b-83b1-dd42dcb18cb4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:03:55.846139 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f68bce-dbf0-485b-83b1-dd42dcb18cb4 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:03:55.899215 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c2c2c3-b45e-4995-8b11-24e02a92c37a None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:03:55.927770 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:03:55.933552 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc1c19c-9119-4755-b33c-b0d145d5400d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:03:55.934372 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc1c19c-9119-4755-b33c-b0d145d5400d None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:03:55.934658 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc1c19c-9119-4755-b33c-b0d145d5400d None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:03:55.934936 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc1c19c-9119-4755-b33c-b0d145d5400d None admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:03:55.935733 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc1c19c-9119-4755-b33c-b0d145d5400d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:03:55.936050 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc1c19c-9119-4755-b33c-b0d145d5400d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:03:55.937445 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc1c19c-9119-4755-b33c-b0d145d5400d None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:03:55.943141 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 39/82] 10.222.0.26 () {58 vars in 1282 bytes} [Thu Jul 14 09:03:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 09:03:55.944016 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f68bce-dbf0-485b-83b1-dd42dcb18cb4 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:03:55.949496 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 44/83] 10.222.0.26 () {56 vars in 1111 bytes} [Thu Jul 14 09:03:55 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:03:55.949496 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc1c19c-9119-4755-b33c-b0d145d5400d None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:03:55.996116 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:03:56.046451 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f68bce-dbf0-485b-83b1-dd42dcb18cb4 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:03:56.046634 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc1c19c-9119-4755-b33c-b0d145d5400d None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:03:56.079818 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3839dff5-27aa-4602-9af2-a99a6c270f49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:03:56.079968 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9cc804-e84d-453a-b7a1-053a1e44c8b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:03:56.080693 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3839dff5-27aa-4602-9af2-a99a6c270f49 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:03:56.080793 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9cc804-e84d-453a-b7a1-053a1e44c8b4 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:03:56.080978 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3839dff5-27aa-4602-9af2-a99a6c270f49 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:03:56.081065 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9cc804-e84d-453a-b7a1-053a1e44c8b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:03:56.081255 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3839dff5-27aa-4602-9af2-a99a6c270f49 None admin] PATH_INFO: `/v3/users/cb553a2f9c234bcbadbe3b9dbdd4a92e` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:03:56.081351 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9cc804-e84d-453a-b7a1-053a1e44c8b4 None admin] PATH_INFO: `/v3/projects/4a4ff416931141ce83908e05cc434590` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:03:56.082472 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3839dff5-27aa-4602-9af2-a99a6c270f49 None admin] RBAC: Authorizing `identity:get_user(user_id=cb553a2f9c234bcbadbe3b9dbdd4a92e)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:03:56.082572 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9cc804-e84d-453a-b7a1-053a1e44c8b4 None admin] RBAC: Authorizing `identity:get_project(project_id=4a4ff416931141ce83908e05cc434590)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:03:56.083971 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3839dff5-27aa-4602-9af2-a99a6c270f49 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:03:56.084062 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9cc804-e84d-453a-b7a1-053a1e44c8b4 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:03:56.085669 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 45/85] 10.222.0.26 () {56 vars in 1186 bytes} [Thu Jul 14 09:03:55 2022] GET /identity/v3/projects/4a4ff416931141ce83908e05cc434590 => generated 298 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:03:56.085863 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 40/85] 10.222.0.26 () {56 vars in 1177 bytes} [Thu Jul 14 09:03:55 2022] GET /identity/v3/users/cb553a2f9c234bcbadbe3b9dbdd4a92e => generated 275 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:03:56.091114 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3839dff5-27aa-4602-9af2-a99a6c270f49 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:03:56.091263 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9cc804-e84d-453a-b7a1-053a1e44c8b4 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:03:56.150179 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:03:56.189062 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9cc804-e84d-453a-b7a1-053a1e44c8b4 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:03:56.190841 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3839dff5-27aa-4602-9af2-a99a6c270f49 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:03:56.207696 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:03:56.222864 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6baacb85-c6e5-4784-9390-48925f18b2ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:03:56.223691 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baacb85-c6e5-4784-9390-48925f18b2ec None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:03:56.223978 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baacb85-c6e5-4784-9390-48925f18b2ec None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:03:56.224255 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baacb85-c6e5-4784-9390-48925f18b2ec None admin] PATH_INFO: `/v3/domains/default` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:03:56.225413 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6baacb85-c6e5-4784-9390-48925f18b2ec None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:03:56.226237 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7802ba-6c2b-405b-8de9-093582ae3a7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:03:56.226865 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6baacb85-c6e5-4784-9390-48925f18b2ec None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:03:56.227111 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7802ba-6c2b-405b-8de9-093582ae3a7b None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:03:56.227422 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7802ba-6c2b-405b-8de9-093582ae3a7b None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:03:56.227699 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7802ba-6c2b-405b-8de9-093582ae3a7b None admin] PATH_INFO: `/v3/projects/4a4ff416931141ce83908e05cc434590` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:03:56.228714 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 46/86] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jul 14 09:03:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:03:56.228896 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7802ba-6c2b-405b-8de9-093582ae3a7b None admin] RBAC: Authorizing `identity:get_project(project_id=4a4ff416931141ce83908e05cc434590)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:03:56.229290 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6baacb85-c6e5-4784-9390-48925f18b2ec None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:03:56.230344 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7802ba-6c2b-405b-8de9-093582ae3a7b None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:03:56.232145 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 41/87] 10.222.0.26 () {56 vars in 1186 bytes} [Thu Jul 14 09:03:56 2022] GET /identity/v3/projects/4a4ff416931141ce83908e05cc434590 => generated 298 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:03:56.232901 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7802ba-6c2b-405b-8de9-093582ae3a7b None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:03:56.328777 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6baacb85-c6e5-4784-9390-48925f18b2ec None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:03:56.333788 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7802ba-6c2b-405b-8de9-093582ae3a7b None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:03:56.357421 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:03:56.363954 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8861f64d-8cbc-415f-9617-a2bc9da5c580 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:03:56.364843 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8861f64d-8cbc-415f-9617-a2bc9da5c580 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:03:56.365132 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8861f64d-8cbc-415f-9617-a2bc9da5c580 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:03:56.365416 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8861f64d-8cbc-415f-9617-a2bc9da5c580 None admin] PATH_INFO: `/v3/users/cb553a2f9c234bcbadbe3b9dbdd4a92e` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:03:56.366638 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8861f64d-8cbc-415f-9617-a2bc9da5c580 None admin] RBAC: Authorizing `identity:get_user(user_id=cb553a2f9c234bcbadbe3b9dbdd4a92e)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:03:56.367024 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ff0178-83c2-4320-bb92-126c719e9c69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:03:56.367704 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ff0178-83c2-4320-bb92-126c719e9c69 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:03:56.367992 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ff0178-83c2-4320-bb92-126c719e9c69 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:03:56.368121 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8861f64d-8cbc-415f-9617-a2bc9da5c580 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:03:56.368272 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ff0178-83c2-4320-bb92-126c719e9c69 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:03:56.369820 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 47/88] 10.222.0.26 () {56 vars in 1177 bytes} [Thu Jul 14 09:03:56 2022] GET /identity/v3/users/cb553a2f9c234bcbadbe3b9dbdd4a92e => generated 275 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:03:56.370580 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8861f64d-8cbc-415f-9617-a2bc9da5c580 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:03:56.419035 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:03:56.436951 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ff0178-83c2-4320-bb92-126c719e9c69 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:03:56.438421 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ff0178-83c2-4320-bb92-126c719e9c69 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:03:56.471374 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8861f64d-8cbc-415f-9617-a2bc9da5c580 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:03:56.504342 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40428ecf-888b-4a46-890f-1aae9810ec23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:03:56.505001 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40428ecf-888b-4a46-890f-1aae9810ec23 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:03:56.505291 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40428ecf-888b-4a46-890f-1aae9810ec23 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:03:56.505581 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40428ecf-888b-4a46-890f-1aae9810ec23 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:03:56.506420 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40428ecf-888b-4a46-890f-1aae9810ec23 None admin] RBAC: Adding query filter params (role.id=d1691322ce6a4edab7ce722edf0f2988, scope.project.id=4a4ff416931141ce83908e05cc434590, user.id=cb553a2f9c234bcbadbe3b9dbdd4a92e) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:03:56.506737 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40428ecf-888b-4a46-890f-1aae9810ec23 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:03:56.508240 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40428ecf-888b-4a46-890f-1aae9810ec23 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:03:56.513710 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 48/89] 10.222.0.26 () {56 vars in 1374 bytes} [Thu Jul 14 09:03:56 2022] GET /identity/v3/role_assignments?user.id=cb553a2f9c234bcbadbe3b9dbdd4a92e&scope.project.id=4a4ff416931141ce83908e05cc434590&role.id=d1691322ce6a4edab7ce722edf0f2988 => generated 259 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:03:56.514466 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40428ecf-888b-4a46-890f-1aae9810ec23 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:03:56.538692 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 42/90] 10.222.0.26 () {58 vars in 1282 bytes} [Thu Jul 14 09:03:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 09:03:56.539699 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ff0178-83c2-4320-bb92-126c719e9c69 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:03:56.597693 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:03:56.640217 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40428ecf-888b-4a46-890f-1aae9810ec23 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:03:56.642370 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:03:56.665020 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ff0178-83c2-4320-bb92-126c719e9c69 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:03:56.678660 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0df3da-b8f9-4904-9598-1efdca54fcd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:03:56.679370 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0df3da-b8f9-4904-9598-1efdca54fcd3 None admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:03:56.679655 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0df3da-b8f9-4904-9598-1efdca54fcd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:03:56.679931 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0df3da-b8f9-4904-9598-1efdca54fcd3 None admin] PATH_INFO: `/v3/users` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:03:56.680657 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0df3da-b8f9-4904-9598-1efdca54fcd3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:03:56.682420 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0df3da-b8f9-4904-9598-1efdca54fcd3 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:03:56.703335 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63dc56f-6a6a-43db-ab05-5c9a348d1954 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:03:56.704030 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63dc56f-6a6a-43db-ab05-5c9a348d1954 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:03:56.704359 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63dc56f-6a6a-43db-ab05-5c9a348d1954 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:03:56.704639 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63dc56f-6a6a-43db-ab05-5c9a348d1954 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:03:56.705483 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63dc56f-6a6a-43db-ab05-5c9a348d1954 None admin] RBAC: Adding query filter params (role.id=7b549d58d1724395b343609d895c80d0, scope.project.id=4a4ff416931141ce83908e05cc434590, user.id=d00c29cc14a043099fbeda47a3389faa) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:03:56.705792 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63dc56f-6a6a-43db-ab05-5c9a348d1954 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:03:56.707279 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63dc56f-6a6a-43db-ab05-5c9a348d1954 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:03:56.719334 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 49/91] 10.222.0.26 () {60 vars in 1132 bytes} [Thu Jul 14 09:03:56 2022] POST /identity/v3/users => generated 289 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 09:03:56.719334 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0df3da-b8f9-4904-9598-1efdca54fcd3 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:03:56.722321 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 43/92] 10.222.0.26 () {56 vars in 1374 bytes} [Thu Jul 14 09:03:56 2022] GET /identity/v3/role_assignments?user.id=d00c29cc14a043099fbeda47a3389faa&scope.project.id=4a4ff416931141ce83908e05cc434590&role.id=7b549d58d1724395b343609d895c80d0 => generated 259 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:03:56.723223 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63dc56f-6a6a-43db-ab05-5c9a348d1954 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:03:56.926702 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0df3da-b8f9-4904-9598-1efdca54fcd3 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:03:56.929968 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63dc56f-6a6a-43db-ab05-5c9a348d1954 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:03:56.956837 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:03:56.973338 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1222ad15-b8a5-4447-a338-97a78bab9c96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:03:56.974625 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1222ad15-b8a5-4447-a338-97a78bab9c96 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:03:56.975678 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1222ad15-b8a5-4447-a338-97a78bab9c96 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:03:56.976167 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1222ad15-b8a5-4447-a338-97a78bab9c96 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:03:56.978097 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a05d7e-0347-4366-bc0f-0cb247e07e71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:03:56.979178 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a05d7e-0347-4366-bc0f-0cb247e07e71 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:03:56.979939 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a05d7e-0347-4366-bc0f-0cb247e07e71 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:03:56.981419 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a05d7e-0347-4366-bc0f-0cb247e07e71 None admin] PATH_INFO: `/v3/projects/240957f502c841f1abce4fa798cf5d70` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:03:56.982869 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a05d7e-0347-4366-bc0f-0cb247e07e71 None admin] RBAC: Authorizing `identity:get_project(project_id=240957f502c841f1abce4fa798cf5d70)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:03:56.984558 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a05d7e-0347-4366-bc0f-0cb247e07e71 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:03:56.987327 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 44/93] 10.222.0.26 () {56 vars in 1186 bytes} [Thu Jul 14 09:03:56 2022] GET /identity/v3/projects/240957f502c841f1abce4fa798cf5d70 => generated 312 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:03:56.993509 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a05d7e-0347-4366-bc0f-0cb247e07e71 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:03:57.020400 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:03:57.074432 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1222ad15-b8a5-4447-a338-97a78bab9c96 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:03:57.076246 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1222ad15-b8a5-4447-a338-97a78bab9c96 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:03:57.127296 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a05d7e-0347-4366-bc0f-0cb247e07e71 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:03:57.172003 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ced4119-5fc8-49d7-adf1-2428caa70d48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:03:57.172916 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ced4119-5fc8-49d7-adf1-2428caa70d48 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:03:57.173772 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ced4119-5fc8-49d7-adf1-2428caa70d48 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:03:57.174194 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ced4119-5fc8-49d7-adf1-2428caa70d48 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:03:57.175503 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ced4119-5fc8-49d7-adf1-2428caa70d48 None admin] RBAC: Adding query filter params (role.id=d1691322ce6a4edab7ce722edf0f2988, scope.project.id=240957f502c841f1abce4fa798cf5d70, user.id=cb553a2f9c234bcbadbe3b9dbdd4a92e) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:03:57.175961 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ced4119-5fc8-49d7-adf1-2428caa70d48 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:03:57.178343 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ced4119-5fc8-49d7-adf1-2428caa70d48 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:03:57.186688 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 45/94] 10.222.0.26 () {56 vars in 1374 bytes} [Thu Jul 14 09:03:56 2022] GET /identity/v3/role_assignments?user.id=cb553a2f9c234bcbadbe3b9dbdd4a92e&scope.project.id=240957f502c841f1abce4fa798cf5d70&role.id=d1691322ce6a4edab7ce722edf0f2988 => generated 259 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:03:57.205012 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 50/95] 10.222.0.26 () {58 vars in 1282 bytes} [Thu Jul 14 09:03:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 493 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 09:03:57.211072 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ced4119-5fc8-49d7-adf1-2428caa70d48 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:03:57.310033 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:03:57.336895 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ced4119-5fc8-49d7-adf1-2428caa70d48 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:03:57.379987 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb697af-b000-4340-a7e3-c44d14392db6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:03:57.380871 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb697af-b000-4340-a7e3-c44d14392db6 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:03:57.381161 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb697af-b000-4340-a7e3-c44d14392db6 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:03:57.381440 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb697af-b000-4340-a7e3-c44d14392db6 None admin] PATH_INFO: `/v3/projects/4a4ff416931141ce83908e05cc434590` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:03:57.383377 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb697af-b000-4340-a7e3-c44d14392db6 None admin] RBAC: Authorizing `identity:get_project(project_id=4a4ff416931141ce83908e05cc434590)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:03:57.384858 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb697af-b000-4340-a7e3-c44d14392db6 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:03:57.386950 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 46/96] 10.222.0.26 () {56 vars in 1186 bytes} [Thu Jul 14 09:03:57 2022] GET /identity/v3/projects/4a4ff416931141ce83908e05cc434590 => generated 298 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:03:57.392610 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1222ad15-b8a5-4447-a338-97a78bab9c96 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:03:57.411851 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:03:57.505880 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1222ad15-b8a5-4447-a338-97a78bab9c96 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:03:57.546372 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc3eea9-0f5c-47ec-82f7-fa2ff16e60b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:03:57.547061 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc3eea9-0f5c-47ec-82f7-fa2ff16e60b9 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:03:57.549923 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc3eea9-0f5c-47ec-82f7-fa2ff16e60b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:03:57.549923 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc3eea9-0f5c-47ec-82f7-fa2ff16e60b9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:03:57.549923 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc3eea9-0f5c-47ec-82f7-fa2ff16e60b9 None admin] RBAC: Adding query filter params (role.id=c4bab4a02e5c4f5f97a024e844a0c072, scope.project.id=4a4ff416931141ce83908e05cc434590, user.id=cb553a2f9c234bcbadbe3b9dbdd4a92e) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:03:57.549923 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc3eea9-0f5c-47ec-82f7-fa2ff16e60b9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:03:57.550338 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc3eea9-0f5c-47ec-82f7-fa2ff16e60b9 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:03:57.556873 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 51/97] 10.222.0.26 () {56 vars in 1374 bytes} [Thu Jul 14 09:03:57 2022] GET /identity/v3/role_assignments?user.id=cb553a2f9c234bcbadbe3b9dbdd4a92e&scope.project.id=4a4ff416931141ce83908e05cc434590&role.id=c4bab4a02e5c4f5f97a024e844a0c072 => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:03:58.962795 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37da0cc-9f13-4a7e-87e1-2a9d74284fae None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:03:58.963327 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37da0cc-9f13-4a7e-87e1-2a9d74284fae None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:03:58.963327 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37da0cc-9f13-4a7e-87e1-2a9d74284fae None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:03:58.964790 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 47/98] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:03:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:03:58.970166 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b6e901-f7d6-44d0-a69f-55783734e612 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:03:58.970558 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b6e901-f7d6-44d0-a69f-55783734e612 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:03:58.970739 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b6e901-f7d6-44d0-a69f-55783734e612 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:03:58.987706 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-87b6e901-f7d6-44d0-a69f-55783734e612 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:03:58.988892 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-87b6e901-f7d6-44d0-a69f-55783734e612 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:03:59.067480 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 52/99] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:03:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 09:03:59.101927 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37da0cc-9f13-4a7e-87e1-2a9d74284fae None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:03:59.229931 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37da0cc-9f13-4a7e-87e1-2a9d74284fae None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:03:59.275818 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f0fe9f-6b7c-454d-84f1-150f7f17240b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:03:59.276620 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f0fe9f-6b7c-454d-84f1-150f7f17240b None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:03:59.276906 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f0fe9f-6b7c-454d-84f1-150f7f17240b None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:03:59.277217 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f0fe9f-6b7c-454d-84f1-150f7f17240b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:03:59.292151 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f0fe9f-6b7c-454d-84f1-150f7f17240b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 09:03:59.299928 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-00f0fe9f-6b7c-454d-84f1-150f7f17240b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 09:03:59.299928 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:03:59.299928 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:03:59.299928 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:03:59.299928 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:03:59.299928 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:03:59.299928 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:03:59.299928 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:03:59.299928 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:03:59.299928 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:03:59.299928 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:03:59.299928 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:03:59.299928 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 09:03:59.299928 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 09:03:59.299928 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 09:03:59.299928 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 09:03:59.300785 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 09:03:59.300785 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 09:03:59.300785 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 09:03:59.300785 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 09:03:59.300785 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:03:59.300785 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:03:59.300785 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:03:59.300785 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:03:59.300785 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:03:59.300785 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:03:59.300785 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:03:59.300785 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 14 09:03:59.300785 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:03:59.300785 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:03:59.300785 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:03:59.300785 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:03:59.302895 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:03:59.302895 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:03:59.302895 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:03:59.302895 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:03:59.302895 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 09:03:59.302895 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 09:03:59.302895 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 09:03:59.302895 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 09:03:59.302895 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 09:03:59.302895 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 09:03:59.302895 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 09:03:59.302895 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 14 09:03:59.302895 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 48/100] 10.222.0.26 () {56 vars in 1099 bytes} [Thu Jul 14 09:03:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:03:59.311592 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b6e901-f7d6-44d0-a69f-55783734e612 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:03:59.337101 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:03:59.463861 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b6e901-f7d6-44d0-a69f-55783734e612 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:03:59.500273 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd47457a-e866-4e1c-b7a7-385861f2523f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:03:59.500874 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd47457a-e866-4e1c-b7a7-385861f2523f None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:03:59.501163 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd47457a-e866-4e1c-b7a7-385861f2523f None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:03:59.501443 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd47457a-e866-4e1c-b7a7-385861f2523f None admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:03:59.502214 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd47457a-e866-4e1c-b7a7-385861f2523f None admin] RBAC: Adding query filter params (name=member) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:03:59.502525 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd47457a-e866-4e1c-b7a7-385861f2523f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:03:59.503987 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd47457a-e866-4e1c-b7a7-385861f2523f None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:03:59.513123 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 53/101] 10.222.0.26 () {56 vars in 1101 bytes} [Thu Jul 14 09:03:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:03:59.518612 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f0fe9f-6b7c-454d-84f1-150f7f17240b None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:03:59.531876 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:03:59.580118 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8db45d-7543-4846-845a-06f69cbf9a34 None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:03:59.580792 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8db45d-7543-4846-845a-06f69cbf9a34 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:03:59.581289 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8db45d-7543-4846-845a-06f69cbf9a34 None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:03:59.582889 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 54/102] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:03:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 09:03:59.588725 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e756a84-a5ed-48e9-95a3-d0540821fc09 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:03:59.589380 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e756a84-a5ed-48e9-95a3-d0540821fc09 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:03:59.589914 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e756a84-a5ed-48e9-95a3-d0540821fc09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:03:59.617425 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-7e756a84-a5ed-48e9-95a3-d0540821fc09 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:03:59.618646 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-7e756a84-a5ed-48e9-95a3-d0540821fc09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:03:59.680105 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f0fe9f-6b7c-454d-84f1-150f7f17240b None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:03:59.737772 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 55/103] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:03:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 09:03:59.743070 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c5b673-1f7b-4d4e-9647-da05c9046ea4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:03:59.744191 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c5b673-1f7b-4d4e-9647-da05c9046ea4 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:03:59.744620 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c5b673-1f7b-4d4e-9647-da05c9046ea4 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:03:59.745039 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c5b673-1f7b-4d4e-9647-da05c9046ea4 None admin] PATH_INFO: `/v3/users/cb553a2f9c234bcbadbe3b9dbdd4a92e` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:03:59.746457 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c5b673-1f7b-4d4e-9647-da05c9046ea4 None admin] RBAC: Authorizing `identity:get_user(user_id=cb553a2f9c234bcbadbe3b9dbdd4a92e)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:03:59.748089 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c5b673-1f7b-4d4e-9647-da05c9046ea4 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:03:59.750021 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 49/104] 10.222.0.26 () {56 vars in 1177 bytes} [Thu Jul 14 09:03:59 2022] GET /identity/v3/users/cb553a2f9c234bcbadbe3b9dbdd4a92e => generated 275 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:03:59.756012 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e756a84-a5ed-48e9-95a3-d0540821fc09 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:03:59.774423 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c5b673-1f7b-4d4e-9647-da05c9046ea4 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:03:59.780247 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:03:59.903237 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:03:59.931172 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e756a84-a5ed-48e9-95a3-d0540821fc09 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:03:59.951380 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c5b673-1f7b-4d4e-9647-da05c9046ea4 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:03:59.978392 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791df382-a0eb-4934-befe-aceaacb169d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:03:59.979337 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791df382-a0eb-4934-befe-aceaacb169d2 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:03:59.979624 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791df382-a0eb-4934-befe-aceaacb169d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:03:59.979900 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791df382-a0eb-4934-befe-aceaacb169d2 None admin] PATH_INFO: `/v3/projects/4a4ff416931141ce83908e05cc434590` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:03:59.981351 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791df382-a0eb-4934-befe-aceaacb169d2 None admin] RBAC: Authorizing `identity:get_project(project_id=4a4ff416931141ce83908e05cc434590)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:03:59.982843 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791df382-a0eb-4934-befe-aceaacb169d2 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:03:59.989819 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 56/105] 10.222.0.26 () {56 vars in 1186 bytes} [Thu Jul 14 09:03:59 2022] GET /identity/v3/projects/4a4ff416931141ce83908e05cc434590 => generated 298 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:03:59.991264 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8668cbb6-0149-4e9a-8126-317751799f74 None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:03:59.991558 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8668cbb6-0149-4e9a-8126-317751799f74 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:03:59.991841 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8668cbb6-0149-4e9a-8126-317751799f74 None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:03:59.993830 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 57/106] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:03:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:03:59.995082 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8668cbb6-0149-4e9a-8126-317751799f74 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:00.071664 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:00.103591 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8668cbb6-0149-4e9a-8126-317751799f74 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:00.103924 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab025247-d657-4168-b85f-05f90efc4bd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:00.104763 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab025247-d657-4168-b85f-05f90efc4bd0 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:00.105116 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab025247-d657-4168-b85f-05f90efc4bd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:00.105400 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab025247-d657-4168-b85f-05f90efc4bd0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:00.140302 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d7ffdb-a0df-4db1-93fd-14922132ea2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:00.141490 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d7ffdb-a0df-4db1-93fd-14922132ea2f None admin] REQUEST_METHOD: `PUT` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:00.141911 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d7ffdb-a0df-4db1-93fd-14922132ea2f None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:00.142324 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d7ffdb-a0df-4db1-93fd-14922132ea2f None admin] PATH_INFO: `/v3/projects/4a4ff416931141ce83908e05cc434590/users/cb553a2f9c234bcbadbe3b9dbdd4a92e/roles/d1691322ce6a4edab7ce722edf0f2988` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:00.144826 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d7ffdb-a0df-4db1-93fd-14922132ea2f None admin] RBAC: Authorizing `identity:create_grant(project_id=4a4ff416931141ce83908e05cc434590, user_id=cb553a2f9c234bcbadbe3b9dbdd4a92e, role_id=d1691322ce6a4edab7ce722edf0f2988)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:00.146477 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d7ffdb-a0df-4db1-93fd-14922132ea2f None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:00.157955 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab025247-d657-4168-b85f-05f90efc4bd0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 09:04:00.173575 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 58/107] 10.222.0.26 () {58 vars in 1439 bytes} [Thu Jul 14 09:03:59 2022] PUT /identity/v3/projects/4a4ff416931141ce83908e05cc434590/users/cb553a2f9c234bcbadbe3b9dbdd4a92e/roles/d1691322ce6a4edab7ce722edf0f2988 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:04:00.175233 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1232644-5778-46cb-b667-cd8d0962f4dc None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:00.177409 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1232644-5778-46cb-b667-cd8d0962f4dc None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:00.177942 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1232644-5778-46cb-b667-cd8d0962f4dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:00.179617 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-ab025247-d657-4168-b85f-05f90efc4bd0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 09:04:00.179617 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:04:00.179617 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:04:00.179617 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:04:00.179617 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:04:00.179617 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:04:00.179617 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:04:00.179617 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:04:00.179617 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:04:00.179617 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:04:00.179617 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:04:00.179617 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:04:00.179617 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 09:04:00.179617 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 09:04:00.179617 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 09:04:00.179617 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 09:04:00.181017 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 09:04:00.181017 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 09:04:00.181017 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 09:04:00.181017 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 09:04:00.181017 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:04:00.181017 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:04:00.181017 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:04:00.181017 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:04:00.181017 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:04:00.181017 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:04:00.181017 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:04:00.181017 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 14 09:04:00.181017 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:04:00.181017 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:04:00.181017 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:04:00.181017 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:04:00.181943 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:04:00.181943 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:04:00.181943 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:04:00.181943 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:04:00.181943 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 09:04:00.181943 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 09:04:00.181943 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 09:04:00.181943 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 09:04:00.181943 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 09:04:00.181943 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 09:04:00.181943 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 09:04:00.181943 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 14 09:04:00.186834 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 50/108] 10.222.0.26 () {56 vars in 1096 bytes} [Thu Jul 14 09:03:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 413 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 09:04:00.193717 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab025247-d657-4168-b85f-05f90efc4bd0 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:00.219933 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-e1232644-5778-46cb-b667-cd8d0962f4dc None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:04:00.222056 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-e1232644-5778-46cb-b667-cd8d0962f4dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:04:00.368866 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 59/109] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:04:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 09:04:00.369978 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2024d0d-173b-4832-a6e4-cf6bff4026a7 None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:00.370279 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2024d0d-173b-4832-a6e4-cf6bff4026a7 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:00.370640 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2024d0d-173b-4832-a6e4-cf6bff4026a7 None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:00.376984 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 60/110] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:04:00 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:04:00.388452 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1030c3-0fd0-4a5a-9f44-540676cac7f2 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:00.388777 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1030c3-0fd0-4a5a-9f44-540676cac7f2 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:00.389015 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1030c3-0fd0-4a5a-9f44-540676cac7f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:00.417672 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-5f1030c3-0fd0-4a5a-9f44-540676cac7f2 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:04:00.419030 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-5f1030c3-0fd0-4a5a-9f44-540676cac7f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:04:00.429893 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab025247-d657-4168-b85f-05f90efc4bd0 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:00.468972 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8961f4fb-8a45-4ece-b8f7-af73c82a0bd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:00.470102 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8961f4fb-8a45-4ece-b8f7-af73c82a0bd0 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:00.470600 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8961f4fb-8a45-4ece-b8f7-af73c82a0bd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:00.471125 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8961f4fb-8a45-4ece-b8f7-af73c82a0bd0 None admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:00.472116 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8961f4fb-8a45-4ece-b8f7-af73c82a0bd0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:04:00.472664 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8961f4fb-8a45-4ece-b8f7-af73c82a0bd0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:00.474303 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8961f4fb-8a45-4ece-b8f7-af73c82a0bd0 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:00.477810 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:00.485724 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 51/111] 10.222.0.26 () {56 vars in 1099 bytes} [Thu Jul 14 09:04:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:04:00.486975 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8961f4fb-8a45-4ece-b8f7-af73c82a0bd0 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:00.494857 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:00.513378 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 61/112] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:04:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 09:04:00.513378 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09de9884-2b38-46ba-8ce0-3a85d5db38fb None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:00.513378 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09de9884-2b38-46ba-8ce0-3a85d5db38fb None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:00.513378 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09de9884-2b38-46ba-8ce0-3a85d5db38fb None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:00.513378 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 62/113] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:04:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:04:00.518394 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09de9884-2b38-46ba-8ce0-3a85d5db38fb None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:00.621674 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8961f4fb-8a45-4ece-b8f7-af73c82a0bd0 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:00.649330 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09de9884-2b38-46ba-8ce0-3a85d5db38fb None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:00.656808 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691f7c07-dc7f-447f-b532-c99367fe5621 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:00.657619 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691f7c07-dc7f-447f-b532-c99367fe5621 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:00.657909 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691f7c07-dc7f-447f-b532-c99367fe5621 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:00.658195 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691f7c07-dc7f-447f-b532-c99367fe5621 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:00.668888 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691f7c07-dc7f-447f-b532-c99367fe5621 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 09:04:00.676684 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-691f7c07-dc7f-447f-b532-c99367fe5621 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 09:04:00.676684 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:04:00.676684 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:04:00.676684 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:04:00.676684 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:04:00.676684 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:04:00.676684 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:04:00.676684 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:04:00.676684 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:04:00.676684 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:04:00.676684 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:04:00.676684 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:04:00.676684 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 09:04:00.676684 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 09:04:00.676684 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 09:04:00.676684 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 09:04:00.677727 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 09:04:00.677727 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 09:04:00.677727 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 09:04:00.677727 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 09:04:00.677727 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:04:00.677727 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:04:00.677727 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:04:00.677727 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:04:00.677727 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:04:00.677727 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:04:00.677727 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:04:00.677727 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 14 09:04:00.677727 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:04:00.677727 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:04:00.677727 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:04:00.677727 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:04:00.678619 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:04:00.678619 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:04:00.678619 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:04:00.678619 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:04:00.678619 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 09:04:00.678619 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 09:04:00.678619 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 09:04:00.678619 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 09:04:00.678619 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 09:04:00.678619 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 09:04:00.678619 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 09:04:00.678619 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 14 09:04:00.678619 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 52/114] 10.222.0.26 () {56 vars in 1099 bytes} [Thu Jul 14 09:04:00 2022] GET /identity/v3/roles/reader => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:04:00.679344 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249c9cd9-7f76-4a2d-84c7-118d268dd313 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:00.679407 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249c9cd9-7f76-4a2d-84c7-118d268dd313 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:00.679674 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249c9cd9-7f76-4a2d-84c7-118d268dd313 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:00.690849 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2133e69f-5e2d-42f8-8044-4a841c5430b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:00.691778 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2133e69f-5e2d-42f8-8044-4a841c5430b8 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:00.692064 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2133e69f-5e2d-42f8-8044-4a841c5430b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:00.692342 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2133e69f-5e2d-42f8-8044-4a841c5430b8 None admin] PATH_INFO: `/v3/users/d00c29cc14a043099fbeda47a3389faa` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:00.693612 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2133e69f-5e2d-42f8-8044-4a841c5430b8 None admin] RBAC: Authorizing `identity:get_user(user_id=d00c29cc14a043099fbeda47a3389faa)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:00.695098 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2133e69f-5e2d-42f8-8044-4a841c5430b8 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:00.701612 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 63/115] 10.222.0.26 () {56 vars in 1177 bytes} [Thu Jul 14 09:04:00 2022] GET /identity/v3/users/d00c29cc14a043099fbeda47a3389faa => generated 247 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:04:00.701612 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2133e69f-5e2d-42f8-8044-4a841c5430b8 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:00.702525 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-249c9cd9-7f76-4a2d-84c7-118d268dd313 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:04:00.703778 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-249c9cd9-7f76-4a2d-84c7-118d268dd313 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:04:00.756457 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:00.766065 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:00.771345 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 53/116] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:04:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 09:04:00.772746 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c2d48f-ed67-42b0-b120-28b824188200 None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:00.773039 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c2d48f-ed67-42b0-b120-28b824188200 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:00.773317 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c2d48f-ed67-42b0-b120-28b824188200 None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:00.774735 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 54/117] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:04:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:04:00.778529 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c2d48f-ed67-42b0-b120-28b824188200 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:00.818835 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2133e69f-5e2d-42f8-8044-4a841c5430b8 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:00.852941 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ec14a4-1e93-43e1-bfac-0531ffd3193b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:00.853725 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ec14a4-1e93-43e1-bfac-0531ffd3193b None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:00.854011 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ec14a4-1e93-43e1-bfac-0531ffd3193b None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:00.854289 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ec14a4-1e93-43e1-bfac-0531ffd3193b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:00.855549 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ec14a4-1e93-43e1-bfac-0531ffd3193b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:00.857037 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ec14a4-1e93-43e1-bfac-0531ffd3193b None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:00.858838 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 64/118] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jul 14 09:04:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:04:00.863239 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ec14a4-1e93-43e1-bfac-0531ffd3193b None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:00.890434 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c2d48f-ed67-42b0-b120-28b824188200 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:00.924951 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4e2ee5-bcdd-4ebb-a7e2-97fff0cee158 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:00.925625 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4e2ee5-bcdd-4ebb-a7e2-97fff0cee158 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:00.925914 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4e2ee5-bcdd-4ebb-a7e2-97fff0cee158 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:00.926194 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4e2ee5-bcdd-4ebb-a7e2-97fff0cee158 None admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:00.926955 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4e2ee5-bcdd-4ebb-a7e2-97fff0cee158 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:04:00.927293 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4e2ee5-bcdd-4ebb-a7e2-97fff0cee158 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:00.928696 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4e2ee5-bcdd-4ebb-a7e2-97fff0cee158 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:00.937143 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 55/119] 10.222.0.26 () {56 vars in 1101 bytes} [Thu Jul 14 09:04:00 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:04:00.938405 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06f35a9-4a3a-4c32-91d1-e37074608177 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:00.938696 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06f35a9-4a3a-4c32-91d1-e37074608177 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:00.938977 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06f35a9-4a3a-4c32-91d1-e37074608177 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:00.955807 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-d06f35a9-4a3a-4c32-91d1-e37074608177 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:04:00.956970 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-d06f35a9-4a3a-4c32-91d1-e37074608177 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:04:00.972034 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ec14a4-1e93-43e1-bfac-0531ffd3193b None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:01.001488 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:01.003425 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:01.013121 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6817d2f8-e49a-43a3-884d-b7a829fe3554 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:01.014233 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6817d2f8-e49a-43a3-884d-b7a829fe3554 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:01.014517 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6817d2f8-e49a-43a3-884d-b7a829fe3554 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:01.015184 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6817d2f8-e49a-43a3-884d-b7a829fe3554 None admin] PATH_INFO: `/v3/projects/4a4ff416931141ce83908e05cc434590` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:01.016609 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6817d2f8-e49a-43a3-884d-b7a829fe3554 None admin] RBAC: Authorizing `identity:get_project(project_id=4a4ff416931141ce83908e05cc434590)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:01.018964 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6817d2f8-e49a-43a3-884d-b7a829fe3554 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:01.021916 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 65/120] 10.222.0.26 () {56 vars in 1186 bytes} [Thu Jul 14 09:04:00 2022] GET /identity/v3/projects/4a4ff416931141ce83908e05cc434590 => generated 298 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:04:01.022707 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6817d2f8-e49a-43a3-884d-b7a829fe3554 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:01.026216 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 56/121] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:04:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 09:04:01.028310 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06f35a9-4a3a-4c32-91d1-e37074608177 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:01.131072 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06f35a9-4a3a-4c32-91d1-e37074608177 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:01.135671 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6817d2f8-e49a-43a3-884d-b7a829fe3554 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:01.167897 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d202e1-696e-48cf-a5db-f55046c0a51d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:01.168841 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d202e1-696e-48cf-a5db-f55046c0a51d None admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:01.168945 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4965bf0-8d5b-4977-be9d-42cf89e0983e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:01.169427 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d202e1-696e-48cf-a5db-f55046c0a51d None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:01.169721 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4965bf0-8d5b-4977-be9d-42cf89e0983e None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:01.170005 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4965bf0-8d5b-4977-be9d-42cf89e0983e None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:01.170281 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4965bf0-8d5b-4977-be9d-42cf89e0983e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:01.170478 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d202e1-696e-48cf-a5db-f55046c0a51d None admin] PATH_INFO: `/v3/projects` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:01.171401 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d202e1-696e-48cf-a5db-f55046c0a51d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:01.173700 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d202e1-696e-48cf-a5db-f55046c0a51d None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:01.181315 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4965bf0-8d5b-4977-be9d-42cf89e0983e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 09:04:01.189956 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-e4965bf0-8d5b-4977-be9d-42cf89e0983e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 09:04:01.189956 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:04:01.189956 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:04:01.189956 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:04:01.189956 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:04:01.189956 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:04:01.189956 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:04:01.189956 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:04:01.189956 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:04:01.189956 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:04:01.189956 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:04:01.189956 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:04:01.189956 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 09:04:01.189956 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 09:04:01.189956 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 09:04:01.189956 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 09:04:01.190820 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 09:04:01.190820 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 09:04:01.190820 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 09:04:01.190820 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 09:04:01.190820 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:04:01.190820 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:04:01.190820 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:04:01.190820 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:04:01.190820 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:04:01.190820 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:04:01.190820 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:04:01.190820 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 14 09:04:01.190820 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:04:01.190820 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:04:01.190820 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:04:01.190820 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:04:01.191647 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:04:01.191647 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:04:01.191647 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:04:01.191647 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:04:01.191647 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 09:04:01.191647 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 09:04:01.191647 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 09:04:01.191647 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 09:04:01.191647 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 09:04:01.191647 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 09:04:01.191647 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 09:04:01.191647 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 14 09:04:01.191647 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 66/122] 10.222.0.26 () {56 vars in 1099 bytes} [Thu Jul 14 09:04:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:04:01.193044 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4965bf0-8d5b-4977-be9d-42cf89e0983e None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:01.207858 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 57/123] 10.222.0.26 () {60 vars in 1141 bytes} [Thu Jul 14 09:04:01 2022] POST /identity/v3/projects => generated 302 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 09:04:01.209043 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d202e1-696e-48cf-a5db-f55046c0a51d None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:01.254338 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:01.263362 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:01.325328 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4965bf0-8d5b-4977-be9d-42cf89e0983e None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:01.352527 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d202e1-696e-48cf-a5db-f55046c0a51d None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:01.379941 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1e8381-afab-460d-8c2a-ba881dbfcc5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:01.380847 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1e8381-afab-460d-8c2a-ba881dbfcc5a None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:01.381132 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1e8381-afab-460d-8c2a-ba881dbfcc5a None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:01.381406 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1e8381-afab-460d-8c2a-ba881dbfcc5a None admin] PATH_INFO: `/v3/users/f27c191891e34eeda402e573588c394c` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:01.397742 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de90c4a9-a6be-48f4-9c30-2f189c54f951 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:01.398689 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1e8381-afab-460d-8c2a-ba881dbfcc5a None admin] RBAC: Authorizing `identity:get_user(user_id=f27c191891e34eeda402e573588c394c)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:01.398796 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de90c4a9-a6be-48f4-9c30-2f189c54f951 None admin] REQUEST_METHOD: `PUT` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:01.399115 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de90c4a9-a6be-48f4-9c30-2f189c54f951 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:01.399473 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de90c4a9-a6be-48f4-9c30-2f189c54f951 None admin] PATH_INFO: `/v3/projects/4a4ff416931141ce83908e05cc434590/users/d00c29cc14a043099fbeda47a3389faa/roles/7b549d58d1724395b343609d895c80d0` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:01.400194 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1e8381-afab-460d-8c2a-ba881dbfcc5a None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:01.404652 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 67/124] 10.222.0.26 () {56 vars in 1177 bytes} [Thu Jul 14 09:04:01 2022] GET /identity/v3/users/f27c191891e34eeda402e573588c394c => generated 289 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:04:01.404774 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de90c4a9-a6be-48f4-9c30-2f189c54f951 None admin] RBAC: Authorizing `identity:create_grant(project_id=4a4ff416931141ce83908e05cc434590, user_id=d00c29cc14a043099fbeda47a3389faa, role_id=7b549d58d1724395b343609d895c80d0)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:01.404849 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1e8381-afab-460d-8c2a-ba881dbfcc5a None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:01.404916 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de90c4a9-a6be-48f4-9c30-2f189c54f951 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:01.422241 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 58/125] 10.222.0.26 () {58 vars in 1439 bytes} [Thu Jul 14 09:04:01 2022] PUT /identity/v3/projects/4a4ff416931141ce83908e05cc434590/users/d00c29cc14a043099fbeda47a3389faa/roles/7b549d58d1724395b343609d895c80d0 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:04:01.423299 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de90c4a9-a6be-48f4-9c30-2f189c54f951 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:01.546660 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:01.557994 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de90c4a9-a6be-48f4-9c30-2f189c54f951 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:01.597733 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eeafb22-f561-4460-9bbf-7fd016ed8500 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:01.598426 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eeafb22-f561-4460-9bbf-7fd016ed8500 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:01.598715 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eeafb22-f561-4460-9bbf-7fd016ed8500 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:01.598997 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eeafb22-f561-4460-9bbf-7fd016ed8500 None admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:01.599858 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eeafb22-f561-4460-9bbf-7fd016ed8500 None admin] RBAC: Adding query filter params (name=member) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:04:01.600346 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eeafb22-f561-4460-9bbf-7fd016ed8500 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:01.601859 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eeafb22-f561-4460-9bbf-7fd016ed8500 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:01.612148 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 59/126] 10.222.0.26 () {56 vars in 1101 bytes} [Thu Jul 14 09:04:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:04:01.613430 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eeafb22-f561-4460-9bbf-7fd016ed8500 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:01.625259 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1e8381-afab-460d-8c2a-ba881dbfcc5a None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:01.658333 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:01.687223 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd089855-41da-4e70-8561-ae6ede93bb15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:01.688034 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd089855-41da-4e70-8561-ae6ede93bb15 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:01.688318 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd089855-41da-4e70-8561-ae6ede93bb15 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:01.688596 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd089855-41da-4e70-8561-ae6ede93bb15 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:01.701690 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd089855-41da-4e70-8561-ae6ede93bb15 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 09:04:01.710216 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-fd089855-41da-4e70-8561-ae6ede93bb15 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 09:04:01.710216 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:04:01.710216 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:04:01.710216 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:04:01.710216 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:04:01.710216 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:04:01.710216 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:04:01.710216 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:04:01.710216 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:04:01.710216 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:04:01.710216 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:04:01.710216 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:04:01.710216 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 09:04:01.710216 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 09:04:01.710216 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 09:04:01.710216 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 09:04:01.711103 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 09:04:01.711103 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 09:04:01.711103 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 09:04:01.711103 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 09:04:01.711103 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:04:01.711103 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:04:01.711103 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:04:01.711103 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:04:01.711103 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:04:01.711103 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:04:01.711103 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:04:01.711103 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 14 09:04:01.711103 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:04:01.711103 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:04:01.711103 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:04:01.711103 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:04:01.711901 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:04:01.711901 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:04:01.711901 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:04:01.711901 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:04:01.711901 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 09:04:01.711901 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 09:04:01.711901 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 09:04:01.711901 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 09:04:01.711901 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 09:04:01.711901 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 09:04:01.711901 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 09:04:01.711901 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 14 09:04:01.713029 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 68/127] 10.222.0.26 () {56 vars in 1114 bytes} [Thu Jul 14 09:04:01 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:04:01.713184 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd089855-41da-4e70-8561-ae6ede93bb15 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:01.741911 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eeafb22-f561-4460-9bbf-7fd016ed8500 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:01.791168 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f4f3d2-b014-463b-8b72-477dfe04380d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:01.792216 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f4f3d2-b014-463b-8b72-477dfe04380d None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:01.792507 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f4f3d2-b014-463b-8b72-477dfe04380d None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:01.793032 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f4f3d2-b014-463b-8b72-477dfe04380d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:01.825487 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:01.826814 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd089855-41da-4e70-8561-ae6ede93bb15 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:01.885530 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7e5abe-6b94-4924-855b-5c2589b323e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:01.887534 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7e5abe-6b94-4924-855b-5c2589b323e3 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:01.887817 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7e5abe-6b94-4924-855b-5c2589b323e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:01.888091 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7e5abe-6b94-4924-855b-5c2589b323e3 None admin] PATH_INFO: `/v3/users/cb553a2f9c234bcbadbe3b9dbdd4a92e` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:01.892492 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7e5abe-6b94-4924-855b-5c2589b323e3 None admin] RBAC: Authorizing `identity:get_user(user_id=cb553a2f9c234bcbadbe3b9dbdd4a92e)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:01.896397 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7e5abe-6b94-4924-855b-5c2589b323e3 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:01.898310 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 69/128] 10.222.0.26 () {56 vars in 1177 bytes} [Thu Jul 14 09:04:01 2022] GET /identity/v3/users/cb553a2f9c234bcbadbe3b9dbdd4a92e => generated 275 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:04:01.901367 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7e5abe-6b94-4924-855b-5c2589b323e3 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:01.902139 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f4f3d2-b014-463b-8b72-477dfe04380d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:01.905838 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f4f3d2-b014-463b-8b72-477dfe04380d None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:01.988891 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:02.077452 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 60/129] 10.222.0.26 () {58 vars in 1282 bytes} [Thu Jul 14 09:04:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 465 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 09:04:02.078127 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f4f3d2-b014-463b-8b72-477dfe04380d None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:02.124947 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7e5abe-6b94-4924-855b-5c2589b323e3 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:02.153273 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:02.196852 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d283e91-3e52-463a-b055-e480fd9b4519 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:02.197539 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d283e91-3e52-463a-b055-e480fd9b4519 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:02.197821 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d283e91-3e52-463a-b055-e480fd9b4519 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:02.198097 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d283e91-3e52-463a-b055-e480fd9b4519 None admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:02.198924 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d283e91-3e52-463a-b055-e480fd9b4519 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:04:02.199248 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d283e91-3e52-463a-b055-e480fd9b4519 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:02.202406 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d283e91-3e52-463a-b055-e480fd9b4519 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:02.220551 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 70/130] 10.222.0.26 () {56 vars in 1111 bytes} [Thu Jul 14 09:04:01 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:04:02.221169 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d283e91-3e52-463a-b055-e480fd9b4519 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:02.243034 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f4f3d2-b014-463b-8b72-477dfe04380d None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:02.308380 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a381e78-5a3b-46b6-b221-7dad449401bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:02.309258 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a381e78-5a3b-46b6-b221-7dad449401bd None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:02.309541 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a381e78-5a3b-46b6-b221-7dad449401bd None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:02.309817 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a381e78-5a3b-46b6-b221-7dad449401bd None admin] PATH_INFO: `/v3/projects/240957f502c841f1abce4fa798cf5d70` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:02.311163 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a381e78-5a3b-46b6-b221-7dad449401bd None admin] RBAC: Authorizing `identity:get_project(project_id=240957f502c841f1abce4fa798cf5d70)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:02.312642 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a381e78-5a3b-46b6-b221-7dad449401bd None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:02.314358 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 61/131] 10.222.0.26 () {56 vars in 1186 bytes} [Thu Jul 14 09:04:02 2022] GET /identity/v3/projects/240957f502c841f1abce4fa798cf5d70 => generated 312 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:04:02.315115 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a381e78-5a3b-46b6-b221-7dad449401bd None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:02.424695 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d283e91-3e52-463a-b055-e480fd9b4519 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:02.428839 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:02.476985 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbd39b3-276e-414e-9863-bc7999f29556 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:02.478505 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbd39b3-276e-414e-9863-bc7999f29556 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:02.478794 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbd39b3-276e-414e-9863-bc7999f29556 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:02.479076 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbd39b3-276e-414e-9863-bc7999f29556 None admin] PATH_INFO: `/v3/projects/4a4ff416931141ce83908e05cc434590` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:02.480976 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbd39b3-276e-414e-9863-bc7999f29556 None admin] RBAC: Authorizing `identity:get_project(project_id=4a4ff416931141ce83908e05cc434590)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:02.482948 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbd39b3-276e-414e-9863-bc7999f29556 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:02.485756 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 71/132] 10.222.0.26 () {56 vars in 1186 bytes} [Thu Jul 14 09:04:02 2022] GET /identity/v3/projects/4a4ff416931141ce83908e05cc434590 => generated 298 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:04:02.486295 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbd39b3-276e-414e-9863-bc7999f29556 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:02.486636 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a381e78-5a3b-46b6-b221-7dad449401bd None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:02.511711 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:02.530346 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68beb0d8-8f34-41e9-b286-a4328e186cfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:02.531612 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68beb0d8-8f34-41e9-b286-a4328e186cfd None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:02.532122 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68beb0d8-8f34-41e9-b286-a4328e186cfd None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:02.532593 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68beb0d8-8f34-41e9-b286-a4328e186cfd None admin] PATH_INFO: `/v3/users/cb553a2f9c234bcbadbe3b9dbdd4a92e` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:02.534539 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68beb0d8-8f34-41e9-b286-a4328e186cfd None admin] RBAC: Authorizing `identity:get_user(user_id=cb553a2f9c234bcbadbe3b9dbdd4a92e)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:02.536236 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68beb0d8-8f34-41e9-b286-a4328e186cfd None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:02.538912 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 62/133] 10.222.0.26 () {56 vars in 1177 bytes} [Thu Jul 14 09:04:02 2022] GET /identity/v3/users/cb553a2f9c234bcbadbe3b9dbdd4a92e => generated 275 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:04:02.541206 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e15beba-d647-4087-ad19-968cebd4be94 None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:02.541696 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e15beba-d647-4087-ad19-968cebd4be94 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:02.542461 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e15beba-d647-4087-ad19-968cebd4be94 None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:02.545083 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 63/134] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:04:02 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:04:02.546304 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e15beba-d647-4087-ad19-968cebd4be94 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:02.618918 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbd39b3-276e-414e-9863-bc7999f29556 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:02.659322 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c408b4dc-c0a1-4580-83c6-4e0a7a2bda02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:02.660341 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c408b4dc-c0a1-4580-83c6-4e0a7a2bda02 None admin] REQUEST_METHOD: `PUT` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:02.660621 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c408b4dc-c0a1-4580-83c6-4e0a7a2bda02 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:02.661093 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c408b4dc-c0a1-4580-83c6-4e0a7a2bda02 None admin] PATH_INFO: `/v3/projects/240957f502c841f1abce4fa798cf5d70/users/cb553a2f9c234bcbadbe3b9dbdd4a92e/roles/d1691322ce6a4edab7ce722edf0f2988` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:02.663274 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c408b4dc-c0a1-4580-83c6-4e0a7a2bda02 None admin] RBAC: Authorizing `identity:create_grant(project_id=240957f502c841f1abce4fa798cf5d70, user_id=cb553a2f9c234bcbadbe3b9dbdd4a92e, role_id=d1691322ce6a4edab7ce722edf0f2988)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:02.664820 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c408b4dc-c0a1-4580-83c6-4e0a7a2bda02 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:02.671529 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e15beba-d647-4087-ad19-968cebd4be94 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:02.682586 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 72/135] 10.222.0.26 () {58 vars in 1439 bytes} [Thu Jul 14 09:04:02 2022] PUT /identity/v3/projects/240957f502c841f1abce4fa798cf5d70/users/cb553a2f9c234bcbadbe3b9dbdd4a92e/roles/d1691322ce6a4edab7ce722edf0f2988 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:04:02.684057 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c408b4dc-c0a1-4580-83c6-4e0a7a2bda02 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:02.718007 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a12f22e-02fe-41ca-9bc1-a554c3c179b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:02.718966 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a12f22e-02fe-41ca-9bc1-a554c3c179b7 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:02.719419 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a12f22e-02fe-41ca-9bc1-a554c3c179b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:02.719856 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a12f22e-02fe-41ca-9bc1-a554c3c179b7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:02.720883 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a12f22e-02fe-41ca-9bc1-a554c3c179b7 None admin] RBAC: Adding query filter params (role.id=9c412baac5914ce6a40ad7d3f0f0a0d6, scope.project.id=4a4ff416931141ce83908e05cc434590, user.id=f27c191891e34eeda402e573588c394c) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:04:02.721363 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a12f22e-02fe-41ca-9bc1-a554c3c179b7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:02.723019 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a12f22e-02fe-41ca-9bc1-a554c3c179b7 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:02.723330 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:02.740119 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 64/136] 10.222.0.26 () {56 vars in 1374 bytes} [Thu Jul 14 09:04:02 2022] GET /identity/v3/role_assignments?user.id=f27c191891e34eeda402e573588c394c&scope.project.id=4a4ff416931141ce83908e05cc434590&role.id=9c412baac5914ce6a40ad7d3f0f0a0d6 => generated 259 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:04:02.741733 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a284ac9d-3a76-4fa3-a0b3-d151dd4fcb45 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:02.742165 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a284ac9d-3a76-4fa3-a0b3-d151dd4fcb45 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:02.742582 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a284ac9d-3a76-4fa3-a0b3-d151dd4fcb45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:02.767298 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-a284ac9d-3a76-4fa3-a0b3-d151dd4fcb45 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:04:02.768752 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-a284ac9d-3a76-4fa3-a0b3-d151dd4fcb45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:04:02.829567 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c408b4dc-c0a1-4580-83c6-4e0a7a2bda02 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:02.847345 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:02.868878 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb61dde3-3fca-466b-960e-278aa64d11ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:02.870015 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb61dde3-3fca-466b-960e-278aa64d11ae None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:02.870473 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb61dde3-3fca-466b-960e-278aa64d11ae None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:02.870780 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb61dde3-3fca-466b-960e-278aa64d11ae None admin] PATH_INFO: `/v3/projects/4a4ff416931141ce83908e05cc434590` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:02.872219 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb61dde3-3fca-466b-960e-278aa64d11ae None admin] RBAC: Authorizing `identity:get_project(project_id=4a4ff416931141ce83908e05cc434590)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:02.874012 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb61dde3-3fca-466b-960e-278aa64d11ae None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:02.876474 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 73/137] 10.222.0.26 () {56 vars in 1186 bytes} [Thu Jul 14 09:04:02 2022] GET /identity/v3/projects/4a4ff416931141ce83908e05cc434590 => generated 298 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:04:02.882088 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb61dde3-3fca-466b-960e-278aa64d11ae None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:02.882609 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 65/138] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:04:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 09:04:02.916501 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a284ac9d-3a76-4fa3-a0b3-d151dd4fcb45 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:03.053490 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:03.056869 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb61dde3-3fca-466b-960e-278aa64d11ae None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:03.081827 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a284ac9d-3a76-4fa3-a0b3-d151dd4fcb45 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:03.095466 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa037a01-e01d-4bd3-9810-08d63042d390 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:03.096375 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa037a01-e01d-4bd3-9810-08d63042d390 None admin] REQUEST_METHOD: `PUT` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:03.096656 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa037a01-e01d-4bd3-9810-08d63042d390 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:03.096933 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa037a01-e01d-4bd3-9810-08d63042d390 None admin] PATH_INFO: `/v3/projects/4a4ff416931141ce83908e05cc434590/users/cb553a2f9c234bcbadbe3b9dbdd4a92e/roles/c4bab4a02e5c4f5f97a024e844a0c072` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:03.099167 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa037a01-e01d-4bd3-9810-08d63042d390 None admin] RBAC: Authorizing `identity:create_grant(project_id=4a4ff416931141ce83908e05cc434590, user_id=cb553a2f9c234bcbadbe3b9dbdd4a92e, role_id=c4bab4a02e5c4f5f97a024e844a0c072)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:03.100681 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa037a01-e01d-4bd3-9810-08d63042d390 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:03.117791 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 74/139] 10.222.0.26 () {58 vars in 1439 bytes} [Thu Jul 14 09:04:02 2022] PUT /identity/v3/projects/4a4ff416931141ce83908e05cc434590/users/cb553a2f9c234bcbadbe3b9dbdd4a92e/roles/c4bab4a02e5c4f5f97a024e844a0c072 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:04:03.153341 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f916d74c-0fa2-4890-a7f6-f9b406711018 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:03.154166 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f916d74c-0fa2-4890-a7f6-f9b406711018 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:03.154448 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f916d74c-0fa2-4890-a7f6-f9b406711018 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:03.154721 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f916d74c-0fa2-4890-a7f6-f9b406711018 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:03.176363 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f916d74c-0fa2-4890-a7f6-f9b406711018 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 09:04:03.185345 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-f916d74c-0fa2-4890-a7f6-f9b406711018 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 09:04:03.185345 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:04:03.185345 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:04:03.185345 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:04:03.185345 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:04:03.185345 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:04:03.185345 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:04:03.185345 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:04:03.185345 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:04:03.185345 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:04:03.185345 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:04:03.185345 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:04:03.185345 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 09:04:03.185345 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 09:04:03.185345 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 09:04:03.185345 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 09:04:03.186340 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 09:04:03.186340 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 09:04:03.186340 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 09:04:03.186340 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 09:04:03.186340 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:04:03.186340 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:04:03.186340 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:04:03.186340 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:04:03.186340 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:04:03.186340 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:04:03.186340 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:04:03.186340 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 14 09:04:03.186340 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:04:03.186340 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:04:03.186340 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:04:03.186340 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:04:03.187133 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:04:03.187133 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:04:03.187133 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:04:03.187133 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:04:03.187133 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 09:04:03.187133 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 09:04:03.187133 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 09:04:03.187133 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 09:04:03.187133 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 09:04:03.187133 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 09:04:03.187133 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 09:04:03.187133 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 14 09:04:03.189885 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 66/140] 10.222.0.26 () {56 vars in 1099 bytes} [Thu Jul 14 09:04:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 09:04:03.196027 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa037a01-e01d-4bd3-9810-08d63042d390 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:03.333162 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa037a01-e01d-4bd3-9810-08d63042d390 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:03.373949 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94752be8-5c4c-4c07-82a7-7eb320b51fe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:03.374637 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94752be8-5c4c-4c07-82a7-7eb320b51fe4 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:03.374918 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94752be8-5c4c-4c07-82a7-7eb320b51fe4 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:03.375206 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94752be8-5c4c-4c07-82a7-7eb320b51fe4 None admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:03.376040 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94752be8-5c4c-4c07-82a7-7eb320b51fe4 None admin] RBAC: Adding query filter params (name=member) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:04:03.376471 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94752be8-5c4c-4c07-82a7-7eb320b51fe4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:03.377865 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94752be8-5c4c-4c07-82a7-7eb320b51fe4 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:03.388759 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 75/141] 10.222.0.26 () {56 vars in 1101 bytes} [Thu Jul 14 09:04:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:04:03.396304 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f916d74c-0fa2-4890-a7f6-f9b406711018 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:03.467431 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:03.541327 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f916d74c-0fa2-4890-a7f6-f9b406711018 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:03.582800 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19ec437-e417-4462-abe2-4f67a4e9d955 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:03.583776 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19ec437-e417-4462-abe2-4f67a4e9d955 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:03.584021 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19ec437-e417-4462-abe2-4f67a4e9d955 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:03.584301 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19ec437-e417-4462-abe2-4f67a4e9d955 None admin] PATH_INFO: `/v3/users/cb553a2f9c234bcbadbe3b9dbdd4a92e` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:03.585525 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19ec437-e417-4462-abe2-4f67a4e9d955 None admin] RBAC: Authorizing `identity:get_user(user_id=cb553a2f9c234bcbadbe3b9dbdd4a92e)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:03.587014 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19ec437-e417-4462-abe2-4f67a4e9d955 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:03.588743 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 67/142] 10.222.0.26 () {56 vars in 1177 bytes} [Thu Jul 14 09:04:03 2022] GET /identity/v3/users/cb553a2f9c234bcbadbe3b9dbdd4a92e => generated 275 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:04:03.594397 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94752be8-5c4c-4c07-82a7-7eb320b51fe4 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:03.625903 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:03.741075 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94752be8-5c4c-4c07-82a7-7eb320b51fe4 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:03.788237 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861a112e-53b4-4126-86cd-d70edb796e2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:03.789269 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861a112e-53b4-4126-86cd-d70edb796e2b None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:03.789759 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861a112e-53b4-4126-86cd-d70edb796e2b None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:03.790221 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861a112e-53b4-4126-86cd-d70edb796e2b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:03.805590 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e77dce-192b-416c-941a-b4e8da13f962 None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:03.805888 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e77dce-192b-416c-941a-b4e8da13f962 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:03.806154 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e77dce-192b-416c-941a-b4e8da13f962 None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:03.807957 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 68/143] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:04:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:04:03.812980 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ac96c7-d859-46cb-a9c0-1e6dd796d45a None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:03.813275 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ac96c7-d859-46cb-a9c0-1e6dd796d45a None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:03.813558 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ac96c7-d859-46cb-a9c0-1e6dd796d45a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:03.831979 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-72ac96c7-d859-46cb-a9c0-1e6dd796d45a None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:04:03.833130 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-72ac96c7-d859-46cb-a9c0-1e6dd796d45a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:04:03.891142 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861a112e-53b4-4126-86cd-d70edb796e2b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:03.892979 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861a112e-53b4-4126-86cd-d70edb796e2b None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:03.955621 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:03.970496 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 69/144] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:04:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 09:04:04.012364 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ac96c7-d859-46cb-a9c0-1e6dd796d45a None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:04.091558 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:04.135499 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 76/145] 10.222.0.26 () {58 vars in 1282 bytes} [Thu Jul 14 09:04:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 541 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 09:04:04.142021 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861a112e-53b4-4126-86cd-d70edb796e2b None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:04.219459 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ac96c7-d859-46cb-a9c0-1e6dd796d45a None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:04.275941 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab0d778-4324-4711-8c88-ea46511ad8fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:04.277108 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab0d778-4324-4711-8c88-ea46511ad8fd None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:04.277614 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab0d778-4324-4711-8c88-ea46511ad8fd None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:04.278096 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab0d778-4324-4711-8c88-ea46511ad8fd None admin] PATH_INFO: `/v3/domains/default` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:04.279545 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab0d778-4324-4711-8c88-ea46511ad8fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:04.281315 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab0d778-4324-4711-8c88-ea46511ad8fd None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:04.283294 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 70/146] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jul 14 09:04:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 09:04:04.295621 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab0d778-4324-4711-8c88-ea46511ad8fd None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:04.334097 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861a112e-53b4-4126-86cd-d70edb796e2b None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:04.373717 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7d9827-5e13-40d5-9351-2fc0a61281ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:04.374646 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7d9827-5e13-40d5-9351-2fc0a61281ba None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:04.374936 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7d9827-5e13-40d5-9351-2fc0a61281ba None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:04.375233 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7d9827-5e13-40d5-9351-2fc0a61281ba None admin] PATH_INFO: `/v3/projects/4a4ff416931141ce83908e05cc434590` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:04.376463 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7d9827-5e13-40d5-9351-2fc0a61281ba None admin] RBAC: Authorizing `identity:get_project(project_id=4a4ff416931141ce83908e05cc434590)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:04.377948 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7d9827-5e13-40d5-9351-2fc0a61281ba None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:04.380148 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 77/147] 10.222.0.26 () {56 vars in 1186 bytes} [Thu Jul 14 09:04:04 2022] GET /identity/v3/projects/4a4ff416931141ce83908e05cc434590 => generated 298 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 09:04:04.381290 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a848faa7-36dd-4158-8351-59187c6a2f22 None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:04.381576 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a848faa7-36dd-4158-8351-59187c6a2f22 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:04.381853 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a848faa7-36dd-4158-8351-59187c6a2f22 None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:04.383095 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 78/148] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:04:04 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:04:04.383545 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:04.388510 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a848faa7-36dd-4158-8351-59187c6a2f22 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:04.432928 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab0d778-4324-4711-8c88-ea46511ad8fd None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:04.444630 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:04.469852 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a2c15b-b4c1-42e6-943b-975bcf676dea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:04.470806 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a2c15b-b4c1-42e6-943b-975bcf676dea None admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:04.471330 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a2c15b-b4c1-42e6-943b-975bcf676dea None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:04.471775 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a2c15b-b4c1-42e6-943b-975bcf676dea None admin] PATH_INFO: `/v3/users` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:04.472693 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a2c15b-b4c1-42e6-943b-975bcf676dea None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:04.474328 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a2c15b-b4c1-42e6-943b-975bcf676dea None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:04.497532 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 71/149] 10.222.0.26 () {60 vars in 1132 bytes} [Thu Jul 14 09:04:04 2022] POST /identity/v3/users => generated 283 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 09:04:04.498043 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a848faa7-36dd-4158-8351-59187c6a2f22 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:04.499528 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3752f1e9-df42-4234-ae64-2bfa8557b4c9 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:04.500124 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3752f1e9-df42-4234-ae64-2bfa8557b4c9 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:04.500587 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3752f1e9-df42-4234-ae64-2bfa8557b4c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:04.520476 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-3752f1e9-df42-4234-ae64-2bfa8557b4c9 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:04:04.522100 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-3752f1e9-df42-4234-ae64-2bfa8557b4c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:04:04.537725 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aaa2f28-2d0e-4ed1-a2b9-f5f3c9269361 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:04.538478 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaa2f28-2d0e-4ed1-a2b9-f5f3c9269361 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:04.538830 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaa2f28-2d0e-4ed1-a2b9-f5f3c9269361 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:04.539176 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaa2f28-2d0e-4ed1-a2b9-f5f3c9269361 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:04.540136 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aaa2f28-2d0e-4ed1-a2b9-f5f3c9269361 None admin] RBAC: Adding query filter params (role.id=d1691322ce6a4edab7ce722edf0f2988, scope.project.id=4a4ff416931141ce83908e05cc434590, user.id=cb553a2f9c234bcbadbe3b9dbdd4a92e) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:04:04.540537 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aaa2f28-2d0e-4ed1-a2b9-f5f3c9269361 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:04.542132 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aaa2f28-2d0e-4ed1-a2b9-f5f3c9269361 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:04.548786 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 79/150] 10.222.0.26 () {56 vars in 1374 bytes} [Thu Jul 14 09:04:04 2022] GET /identity/v3/role_assignments?user.id=cb553a2f9c234bcbadbe3b9dbdd4a92e&scope.project.id=4a4ff416931141ce83908e05cc434590&role.id=d1691322ce6a4edab7ce722edf0f2988 => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:04:04.602156 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 72/151] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:04:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 09:04:04.625866 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aaa2f28-2d0e-4ed1-a2b9-f5f3c9269361 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:04.697346 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:04.759359 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aaa2f28-2d0e-4ed1-a2b9-f5f3c9269361 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:04.795989 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad68cb1-3d81-41a1-9ae0-b037b7d5af34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:04.796803 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad68cb1-3d81-41a1-9ae0-b037b7d5af34 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:04.797082 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad68cb1-3d81-41a1-9ae0-b037b7d5af34 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:04.797349 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad68cb1-3d81-41a1-9ae0-b037b7d5af34 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:04.809554 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad68cb1-3d81-41a1-9ae0-b037b7d5af34 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 09:04:04.817362 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-aad68cb1-3d81-41a1-9ae0-b037b7d5af34 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 09:04:04.817362 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:04:04.817362 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:04:04.817362 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:04:04.817362 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:04:04.817362 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:04:04.817362 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:04:04.817362 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:04:04.817362 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:04:04.817362 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:04:04.817362 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:04:04.817362 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:04:04.817362 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 09:04:04.817362 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 09:04:04.817362 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 09:04:04.817362 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 09:04:04.818276 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 09:04:04.818276 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 09:04:04.818276 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 09:04:04.818276 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 09:04:04.818276 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:04:04.818276 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:04:04.818276 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:04:04.818276 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:04:04.818276 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:04:04.818276 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:04:04.818276 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:04:04.818276 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 14 09:04:04.818276 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:04:04.818276 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:04:04.818276 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:04:04.818276 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:04:04.819051 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:04:04.819051 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:04:04.819051 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:04:04.819051 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:04:04.819051 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 09:04:04.819051 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 09:04:04.819051 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 09:04:04.819051 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 09:04:04.819051 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 09:04:04.819051 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 09:04:04.819051 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 09:04:04.819051 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 14 09:04:04.819051 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 80/152] 10.222.0.26 () {56 vars in 1096 bytes} [Thu Jul 14 09:04:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:04:04.825928 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3752f1e9-df42-4234-ae64-2bfa8557b4c9 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:04.925757 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:04.934625 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3752f1e9-df42-4234-ae64-2bfa8557b4c9 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:04.971958 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f721f8-9ac6-4123-a540-842a263b446e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:04.972636 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f721f8-9ac6-4123-a540-842a263b446e None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:04.972917 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f721f8-9ac6-4123-a540-842a263b446e None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:04.973239 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f721f8-9ac6-4123-a540-842a263b446e None admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:04.974058 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f721f8-9ac6-4123-a540-842a263b446e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:04:04.974360 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f721f8-9ac6-4123-a540-842a263b446e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:04.975785 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f721f8-9ac6-4123-a540-842a263b446e None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:04.984714 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 73/153] 10.222.0.26 () {56 vars in 1099 bytes} [Thu Jul 14 09:04:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:04:04.990216 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad68cb1-3d81-41a1-9ae0-b037b7d5af34 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:05.100212 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad68cb1-3d81-41a1-9ae0-b037b7d5af34 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:05.117262 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:05.143268 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c689f62d-4be2-4d49-837d-c39b934bf70f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:05.144156 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c689f62d-4be2-4d49-837d-c39b934bf70f None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:05.144435 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c689f62d-4be2-4d49-837d-c39b934bf70f None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:05.144705 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c689f62d-4be2-4d49-837d-c39b934bf70f None admin] PATH_INFO: `/v3/users/d00c29cc14a043099fbeda47a3389faa` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:05.147887 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c689f62d-4be2-4d49-837d-c39b934bf70f None admin] RBAC: Authorizing `identity:get_user(user_id=d00c29cc14a043099fbeda47a3389faa)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:05.149345 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c689f62d-4be2-4d49-837d-c39b934bf70f None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:05.151136 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 81/154] 10.222.0.26 () {56 vars in 1177 bytes} [Thu Jul 14 09:04:04 2022] GET /identity/v3/users/d00c29cc14a043099fbeda47a3389faa => generated 247 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:04:05.156713 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f721f8-9ac6-4123-a540-842a263b446e None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:05.261214 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f721f8-9ac6-4123-a540-842a263b446e None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:05.297310 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62619cb7-e54a-49f4-993f-bbde53b6a4eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:05.297963 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62619cb7-e54a-49f4-993f-bbde53b6a4eb None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:05.298245 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62619cb7-e54a-49f4-993f-bbde53b6a4eb None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:05.298565 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62619cb7-e54a-49f4-993f-bbde53b6a4eb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:05.327991 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:05.371603 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62619cb7-e54a-49f4-993f-bbde53b6a4eb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:05.373072 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62619cb7-e54a-49f4-993f-bbde53b6a4eb None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:05.475346 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 74/155] 10.222.0.26 () {58 vars in 1282 bytes} [Thu Jul 14 09:04:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 09:04:05.480975 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c689f62d-4be2-4d49-837d-c39b934bf70f None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:05.601977 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c689f62d-4be2-4d49-837d-c39b934bf70f None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:05.639842 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346097c3-5850-414d-9e29-7c2e35023e4b None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:05.640142 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346097c3-5850-414d-9e29-7c2e35023e4b None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:05.640410 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346097c3-5850-414d-9e29-7c2e35023e4b None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:05.641803 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 75/156] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:04:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 09:04:05.646842 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f26379-d395-4750-a312-022ee5beca46 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:05.647131 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f26379-d395-4750-a312-022ee5beca46 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:05.647266 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f896d5-d894-47a7-8e68-4c368a0e3773 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:05.647475 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f26379-d395-4750-a312-022ee5beca46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:05.648148 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f896d5-d894-47a7-8e68-4c368a0e3773 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:05.648425 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f896d5-d894-47a7-8e68-4c368a0e3773 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:05.648692 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f896d5-d894-47a7-8e68-4c368a0e3773 None admin] PATH_INFO: `/v3/projects/4a4ff416931141ce83908e05cc434590` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:05.651447 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f896d5-d894-47a7-8e68-4c368a0e3773 None admin] RBAC: Authorizing `identity:get_project(project_id=4a4ff416931141ce83908e05cc434590)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:05.652968 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f896d5-d894-47a7-8e68-4c368a0e3773 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:05.655921 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 82/157] 10.222.0.26 () {56 vars in 1186 bytes} [Thu Jul 14 09:04:05 2022] GET /identity/v3/projects/4a4ff416931141ce83908e05cc434590 => generated 298 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:04:05.661645 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f896d5-d894-47a7-8e68-4c368a0e3773 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:05.685376 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-f2f26379-d395-4750-a312-022ee5beca46 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:04:05.686598 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-f2f26379-d395-4750-a312-022ee5beca46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:04:05.752733 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:05.761106 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:05.777381 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 76/158] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:04:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 09:04:05.778736 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78445a38-d42e-4db2-9ac3-8c877faf45d7 None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:05.779024 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78445a38-d42e-4db2-9ac3-8c877faf45d7 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:05.779341 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78445a38-d42e-4db2-9ac3-8c877faf45d7 None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:05.780625 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 77/159] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:04:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:04:05.786419 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30538a5-1ede-4c73-81f0-b42d036e885b None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:05.786705 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30538a5-1ede-4c73-81f0-b42d036e885b None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:05.786984 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30538a5-1ede-4c73-81f0-b42d036e885b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:05.812358 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-c30538a5-1ede-4c73-81f0-b42d036e885b None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:04:05.813528 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-c30538a5-1ede-4c73-81f0-b42d036e885b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:04:05.819422 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f896d5-d894-47a7-8e68-4c368a0e3773 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:05.862836 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c9a08e-576f-471c-b798-536890dbe16f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:05.863826 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c9a08e-576f-471c-b798-536890dbe16f None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:05.864272 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c9a08e-576f-471c-b798-536890dbe16f None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:05.864691 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c9a08e-576f-471c-b798-536890dbe16f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:05.865673 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c9a08e-576f-471c-b798-536890dbe16f None admin] RBAC: Adding query filter params (role.id=7b549d58d1724395b343609d895c80d0, scope.project.id=4a4ff416931141ce83908e05cc434590, user.id=d00c29cc14a043099fbeda47a3389faa) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:04:05.866118 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c9a08e-576f-471c-b798-536890dbe16f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:05.867741 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c9a08e-576f-471c-b798-536890dbe16f None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:05.874639 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 83/160] 10.222.0.26 () {56 vars in 1374 bytes} [Thu Jul 14 09:04:05 2022] GET /identity/v3/role_assignments?user.id=d00c29cc14a043099fbeda47a3389faa&scope.project.id=4a4ff416931141ce83908e05cc434590&role.id=7b549d58d1724395b343609d895c80d0 => generated 608 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 09:04:05.880463 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c9a08e-576f-471c-b798-536890dbe16f None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:05.897297 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 78/161] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:04:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 09:04:05.936686 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30538a5-1ede-4c73-81f0-b42d036e885b None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:06.052193 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c9a08e-576f-471c-b798-536890dbe16f None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:06.075738 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:06.093163 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b070a10a-5374-4d45-be83-9291d5aa3304 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:06.094434 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b070a10a-5374-4d45-be83-9291d5aa3304 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:06.094909 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b070a10a-5374-4d45-be83-9291d5aa3304 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:06.095407 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b070a10a-5374-4d45-be83-9291d5aa3304 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:06.103548 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30538a5-1ede-4c73-81f0-b42d036e885b None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:06.108211 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b070a10a-5374-4d45-be83-9291d5aa3304 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 09:04:06.116534 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-b070a10a-5374-4d45-be83-9291d5aa3304 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 09:04:06.116534 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:04:06.116534 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:04:06.116534 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:04:06.116534 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:04:06.116534 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:04:06.116534 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:04:06.116534 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:04:06.116534 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:04:06.116534 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:04:06.116534 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:04:06.116534 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:04:06.116534 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 09:04:06.116534 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 09:04:06.116534 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 09:04:06.116534 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 09:04:06.117458 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 09:04:06.117458 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 09:04:06.117458 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 09:04:06.117458 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 09:04:06.117458 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:04:06.117458 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:04:06.117458 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:04:06.117458 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:04:06.117458 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:04:06.117458 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:04:06.117458 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:04:06.117458 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 14 09:04:06.117458 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:04:06.117458 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:04:06.117458 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:04:06.117458 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:04:06.118307 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:04:06.118307 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:04:06.118307 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:04:06.118307 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:04:06.118307 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 09:04:06.118307 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 09:04:06.118307 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 09:04:06.118307 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 09:04:06.118307 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 09:04:06.118307 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 09:04:06.118307 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 09:04:06.118307 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 14 09:04:06.120892 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 84/162] 10.222.0.26 () {56 vars in 1099 bytes} [Thu Jul 14 09:04:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:04:06.122529 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f58ca7-216d-461e-bf33-6f0c43ab3268 None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:06.123000 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f58ca7-216d-461e-bf33-6f0c43ab3268 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:06.123541 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f58ca7-216d-461e-bf33-6f0c43ab3268 None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:06.125398 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 85/163] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:04:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:04:06.130650 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9a6db5-65bd-4316-bbf8-863e52750922 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:06.131357 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9a6db5-65bd-4316-bbf8-863e52750922 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:06.131819 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9a6db5-65bd-4316-bbf8-863e52750922 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:06.136726 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:06.154220 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-3f9a6db5-65bd-4316-bbf8-863e52750922 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:04:06.155800 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-3f9a6db5-65bd-4316-bbf8-863e52750922 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:04:06.179771 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfec9e6e-000e-4c73-a929-809e4844fc30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:06.180586 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfec9e6e-000e-4c73-a929-809e4844fc30 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:06.180870 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfec9e6e-000e-4c73-a929-809e4844fc30 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:06.181146 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfec9e6e-000e-4c73-a929-809e4844fc30 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:06.201389 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfec9e6e-000e-4c73-a929-809e4844fc30 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 09:04:06.215602 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-bfec9e6e-000e-4c73-a929-809e4844fc30 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 09:04:06.215602 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:04:06.215602 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:04:06.215602 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:04:06.215602 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:04:06.215602 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:04:06.215602 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:04:06.215602 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:04:06.215602 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:04:06.215602 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:04:06.215602 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:04:06.215602 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:04:06.215602 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 09:04:06.215602 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 09:04:06.215602 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 09:04:06.215602 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 09:04:06.216571 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 09:04:06.216571 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 09:04:06.216571 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 09:04:06.216571 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 09:04:06.216571 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:04:06.216571 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:04:06.216571 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:04:06.216571 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:04:06.216571 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:04:06.216571 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:04:06.216571 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:04:06.216571 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 14 09:04:06.216571 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:04:06.216571 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:04:06.216571 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:04:06.216571 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:04:06.217344 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:04:06.217344 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:04:06.217344 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:04:06.217344 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:04:06.217344 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 09:04:06.217344 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 09:04:06.217344 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 09:04:06.217344 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 09:04:06.217344 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 09:04:06.217344 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 09:04:06.217344 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 09:04:06.217344 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 14 09:04:06.224968 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 79/164] 10.222.0.26 () {56 vars in 1099 bytes} [Thu Jul 14 09:04:05 2022] GET /identity/v3/roles/reader => generated 84 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 09:04:06.225617 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfec9e6e-000e-4c73-a929-809e4844fc30 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:06.269884 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 86/165] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:04:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 09:04:06.271445 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9a6db5-65bd-4316-bbf8-863e52750922 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:06.418244 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfec9e6e-000e-4c73-a929-809e4844fc30 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:06.465277 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9a6db5-65bd-4316-bbf8-863e52750922 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:06.475285 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:06.476925 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412dde89-1dab-4c1a-9f08-59d56bc29195 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:06.477614 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412dde89-1dab-4c1a-9f08-59d56bc29195 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:06.478510 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412dde89-1dab-4c1a-9f08-59d56bc29195 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:06.478510 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412dde89-1dab-4c1a-9f08-59d56bc29195 None admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:06.479000 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412dde89-1dab-4c1a-9f08-59d56bc29195 None admin] RBAC: Adding query filter params (name=member) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:04:06.479417 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412dde89-1dab-4c1a-9f08-59d56bc29195 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:06.480841 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412dde89-1dab-4c1a-9f08-59d56bc29195 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:06.497555 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 80/166] 10.222.0.26 () {56 vars in 1101 bytes} [Thu Jul 14 09:04:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:04:06.499285 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412dde89-1dab-4c1a-9f08-59d56bc29195 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:06.524015 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbc38b3-e600-49a5-bca7-39f633ec8413 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:06.524709 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbc38b3-e600-49a5-bca7-39f633ec8413 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:06.525267 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbc38b3-e600-49a5-bca7-39f633ec8413 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:06.525391 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbc38b3-e600-49a5-bca7-39f633ec8413 None admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:06.526248 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbc38b3-e600-49a5-bca7-39f633ec8413 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:04:06.526554 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbc38b3-e600-49a5-bca7-39f633ec8413 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:06.528075 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbc38b3-e600-49a5-bca7-39f633ec8413 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:06.537481 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 87/167] 10.222.0.26 () {56 vars in 1101 bytes} [Thu Jul 14 09:04:06 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:04:06.538194 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbc38b3-e600-49a5-bca7-39f633ec8413 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:06.599538 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:06.678121 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbc38b3-e600-49a5-bca7-39f633ec8413 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:06.718794 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65552530-807e-404d-81b4-86ac159910f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:06.719723 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65552530-807e-404d-81b4-86ac159910f9 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:06.719999 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65552530-807e-404d-81b4-86ac159910f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:06.720267 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65552530-807e-404d-81b4-86ac159910f9 None admin] PATH_INFO: `/v3/users/cb553a2f9c234bcbadbe3b9dbdd4a92e` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:06.722133 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65552530-807e-404d-81b4-86ac159910f9 None admin] RBAC: Authorizing `identity:get_user(user_id=cb553a2f9c234bcbadbe3b9dbdd4a92e)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:06.723615 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65552530-807e-404d-81b4-86ac159910f9 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:06.726541 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 88/168] 10.222.0.26 () {56 vars in 1177 bytes} [Thu Jul 14 09:04:06 2022] GET /identity/v3/users/cb553a2f9c234bcbadbe3b9dbdd4a92e => generated 275 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:04:06.727339 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65552530-807e-404d-81b4-86ac159910f9 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:06.738004 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412dde89-1dab-4c1a-9f08-59d56bc29195 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:06.797379 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:06.814131 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6214bf0a-5400-4a88-9bb8-0c0c6c9f7526 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:06.814960 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6214bf0a-5400-4a88-9bb8-0c0c6c9f7526 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:06.815261 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6214bf0a-5400-4a88-9bb8-0c0c6c9f7526 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:06.815527 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6214bf0a-5400-4a88-9bb8-0c0c6c9f7526 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:06.835467 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6214bf0a-5400-4a88-9bb8-0c0c6c9f7526 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 09:04:06.844222 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-6214bf0a-5400-4a88-9bb8-0c0c6c9f7526 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 09:04:06.844222 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:04:06.844222 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:04:06.844222 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:04:06.844222 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:04:06.844222 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:04:06.844222 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:04:06.844222 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:04:06.844222 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:04:06.844222 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:04:06.844222 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:04:06.844222 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:04:06.844222 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 09:04:06.844222 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 09:04:06.844222 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 09:04:06.844222 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 09:04:06.845102 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 09:04:06.845102 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 09:04:06.845102 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 09:04:06.845102 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 09:04:06.845102 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:04:06.845102 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:04:06.845102 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:04:06.845102 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:04:06.845102 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:04:06.845102 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:04:06.845102 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:04:06.845102 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 14 09:04:06.845102 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:04:06.845102 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:04:06.845102 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:04:06.845102 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:04:06.845896 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:04:06.845896 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:04:06.845896 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:04:06.845896 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:04:06.845896 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 09:04:06.845896 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 09:04:06.845896 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 09:04:06.845896 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 09:04:06.845896 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 09:04:06.845896 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 09:04:06.845896 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 09:04:06.845896 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 14 09:04:06.848564 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 81/169] 10.222.0.26 () {56 vars in 1114 bytes} [Thu Jul 14 09:04:06 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 350 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:04:06.849587 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6214bf0a-5400-4a88-9bb8-0c0c6c9f7526 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:06.906079 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65552530-807e-404d-81b4-86ac159910f9 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:06.951235 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6214bf0a-5400-4a88-9bb8-0c0c6c9f7526 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:06.961890 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:06.976421 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c667b7fd-cee6-47bc-8ad8-d941919839f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:06.977389 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c667b7fd-cee6-47bc-8ad8-d941919839f3 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:06.977733 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c667b7fd-cee6-47bc-8ad8-d941919839f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:06.978095 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c667b7fd-cee6-47bc-8ad8-d941919839f3 None admin] PATH_INFO: `/v3/users/f27c191891e34eeda402e573588c394c` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:06.979404 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c667b7fd-cee6-47bc-8ad8-d941919839f3 None admin] RBAC: Authorizing `identity:get_user(user_id=f27c191891e34eeda402e573588c394c)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:06.982092 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c667b7fd-cee6-47bc-8ad8-d941919839f3 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:06.987939 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 89/170] 10.222.0.26 () {56 vars in 1177 bytes} [Thu Jul 14 09:04:06 2022] GET /identity/v3/users/f27c191891e34eeda402e573588c394c => generated 289 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:04:06.991959 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c667b7fd-cee6-47bc-8ad8-d941919839f3 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:07.022627 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c373aa67-177e-48cb-be7c-cdb426161747 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:07.024412 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c373aa67-177e-48cb-be7c-cdb426161747 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:07.024696 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c373aa67-177e-48cb-be7c-cdb426161747 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:07.025061 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c373aa67-177e-48cb-be7c-cdb426161747 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:07.159779 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c667b7fd-cee6-47bc-8ad8-d941919839f3 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:07.163512 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:07.179735 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c373aa67-177e-48cb-be7c-cdb426161747 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:07.181277 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c373aa67-177e-48cb-be7c-cdb426161747 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:07.227061 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baabab57-e346-4589-bdd0-c5f82c28fd6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:07.227776 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baabab57-e346-4589-bdd0-c5f82c28fd6c None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:07.228091 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baabab57-e346-4589-bdd0-c5f82c28fd6c None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:07.228370 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baabab57-e346-4589-bdd0-c5f82c28fd6c None admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:07.229130 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baabab57-e346-4589-bdd0-c5f82c28fd6c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:04:07.229525 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baabab57-e346-4589-bdd0-c5f82c28fd6c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:07.230934 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baabab57-e346-4589-bdd0-c5f82c28fd6c None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:07.245885 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 90/171] 10.222.0.26 () {56 vars in 1111 bytes} [Thu Jul 14 09:04:06 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:04:07.251397 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baabab57-e346-4589-bdd0-c5f82c28fd6c None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:07.341905 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:07.346258 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 82/172] 10.222.0.26 () {58 vars in 1282 bytes} [Thu Jul 14 09:04:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 497 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 09:04:07.347094 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c373aa67-177e-48cb-be7c-cdb426161747 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:07.459454 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baabab57-e346-4589-bdd0-c5f82c28fd6c None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:07.511517 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c373aa67-177e-48cb-be7c-cdb426161747 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:07.516334 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e58342-8427-45b4-9bd7-8b0d5c4f5171 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:07.517205 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e58342-8427-45b4-9bd7-8b0d5c4f5171 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:07.517480 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e58342-8427-45b4-9bd7-8b0d5c4f5171 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:07.517745 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e58342-8427-45b4-9bd7-8b0d5c4f5171 None admin] PATH_INFO: `/v3/projects/4a4ff416931141ce83908e05cc434590` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:07.520550 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e58342-8427-45b4-9bd7-8b0d5c4f5171 None admin] RBAC: Authorizing `identity:get_project(project_id=4a4ff416931141ce83908e05cc434590)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:07.522016 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e58342-8427-45b4-9bd7-8b0d5c4f5171 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:07.523780 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 91/173] 10.222.0.26 () {56 vars in 1186 bytes} [Thu Jul 14 09:04:07 2022] GET /identity/v3/projects/4a4ff416931141ce83908e05cc434590 => generated 298 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:04:07.524533 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e58342-8427-45b4-9bd7-8b0d5c4f5171 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:07.586897 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eee82f5-bd2b-43d2-abc9-b6272cfc1429 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:07.587838 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eee82f5-bd2b-43d2-abc9-b6272cfc1429 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:07.588114 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eee82f5-bd2b-43d2-abc9-b6272cfc1429 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:07.588383 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eee82f5-bd2b-43d2-abc9-b6272cfc1429 None admin] PATH_INFO: `/v3/users/cb553a2f9c234bcbadbe3b9dbdd4a92e` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:07.589717 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eee82f5-bd2b-43d2-abc9-b6272cfc1429 None admin] RBAC: Authorizing `identity:get_user(user_id=cb553a2f9c234bcbadbe3b9dbdd4a92e)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:07.591180 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eee82f5-bd2b-43d2-abc9-b6272cfc1429 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:07.592845 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 83/174] 10.222.0.26 () {56 vars in 1177 bytes} [Thu Jul 14 09:04:07 2022] GET /identity/v3/users/cb553a2f9c234bcbadbe3b9dbdd4a92e => generated 275 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:04:07.596087 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eee82f5-bd2b-43d2-abc9-b6272cfc1429 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:07.627579 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:07.688315 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e58342-8427-45b4-9bd7-8b0d5c4f5171 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:07.730840 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:07.750569 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd12bc18-e0c8-4da8-8553-2b0e138631c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:07.751756 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd12bc18-e0c8-4da8-8553-2b0e138631c8 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:07.752183 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd12bc18-e0c8-4da8-8553-2b0e138631c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:07.752672 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd12bc18-e0c8-4da8-8553-2b0e138631c8 None admin] PATH_INFO: `/v3/projects/240957f502c841f1abce4fa798cf5d70` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:07.754114 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd12bc18-e0c8-4da8-8553-2b0e138631c8 None admin] RBAC: Authorizing `identity:get_project(project_id=240957f502c841f1abce4fa798cf5d70)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:07.755784 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd12bc18-e0c8-4da8-8553-2b0e138631c8 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:07.757809 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 92/175] 10.222.0.26 () {56 vars in 1186 bytes} [Thu Jul 14 09:04:07 2022] GET /identity/v3/projects/240957f502c841f1abce4fa798cf5d70 => generated 312 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:04:07.758817 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd12bc18-e0c8-4da8-8553-2b0e138631c8 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:07.768477 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eee82f5-bd2b-43d2-abc9-b6272cfc1429 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:07.804751 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90df9ef6-603f-4d4a-8a4b-4af0e3158a5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:07.805840 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90df9ef6-603f-4d4a-8a4b-4af0e3158a5c None admin] REQUEST_METHOD: `PUT` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:07.806055 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90df9ef6-603f-4d4a-8a4b-4af0e3158a5c None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:07.806323 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90df9ef6-603f-4d4a-8a4b-4af0e3158a5c None admin] PATH_INFO: `/v3/projects/4a4ff416931141ce83908e05cc434590/users/f27c191891e34eeda402e573588c394c/roles/9c412baac5914ce6a40ad7d3f0f0a0d6` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:07.808368 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90df9ef6-603f-4d4a-8a4b-4af0e3158a5c None admin] RBAC: Authorizing `identity:create_grant(project_id=4a4ff416931141ce83908e05cc434590, user_id=f27c191891e34eeda402e573588c394c, role_id=9c412baac5914ce6a40ad7d3f0f0a0d6)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:07.809910 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90df9ef6-603f-4d4a-8a4b-4af0e3158a5c None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:07.825908 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 84/176] 10.222.0.26 () {58 vars in 1439 bytes} [Thu Jul 14 09:04:07 2022] PUT /identity/v3/projects/4a4ff416931141ce83908e05cc434590/users/f27c191891e34eeda402e573588c394c/roles/9c412baac5914ce6a40ad7d3f0f0a0d6 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:04:07.827634 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccdaa95-1f03-40e1-8742-92e9491499b0 None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:07.827924 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccdaa95-1f03-40e1-8742-92e9491499b0 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:07.828201 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccdaa95-1f03-40e1-8742-92e9491499b0 None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:07.834890 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 85/177] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:04:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:04:07.834890 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccdaa95-1f03-40e1-8742-92e9491499b0 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:07.880868 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:07.887098 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd12bc18-e0c8-4da8-8553-2b0e138631c8 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:07.963339 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7022c0-0a58-4e47-ab44-40583397b688 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:07.964394 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7022c0-0a58-4e47-ab44-40583397b688 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:07.964899 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7022c0-0a58-4e47-ab44-40583397b688 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:07.965400 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7022c0-0a58-4e47-ab44-40583397b688 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:07.967592 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccdaa95-1f03-40e1-8742-92e9491499b0 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:08.003641 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:08.009033 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60617153-bd0d-4425-be9b-61444d484e7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:08.014472 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60617153-bd0d-4425-be9b-61444d484e7d None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:08.014472 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60617153-bd0d-4425-be9b-61444d484e7d None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:08.014472 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60617153-bd0d-4425-be9b-61444d484e7d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:08.014472 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60617153-bd0d-4425-be9b-61444d484e7d None admin] RBAC: Adding query filter params (role.id=d1691322ce6a4edab7ce722edf0f2988, scope.project.id=240957f502c841f1abce4fa798cf5d70, user.id=cb553a2f9c234bcbadbe3b9dbdd4a92e) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:04:08.014472 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60617153-bd0d-4425-be9b-61444d484e7d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:08.014472 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60617153-bd0d-4425-be9b-61444d484e7d None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:08.021784 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 86/178] 10.222.0.26 () {56 vars in 1374 bytes} [Thu Jul 14 09:04:07 2022] GET /identity/v3/role_assignments?user.id=cb553a2f9c234bcbadbe3b9dbdd4a92e&scope.project.id=240957f502c841f1abce4fa798cf5d70&role.id=d1691322ce6a4edab7ce722edf0f2988 => generated 608 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:04:08.023659 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00cf304-df34-4ad7-87f7-62fb84ed915b None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:08.023937 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00cf304-df34-4ad7-87f7-62fb84ed915b None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:08.024569 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00cf304-df34-4ad7-87f7-62fb84ed915b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:08.045503 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-a00cf304-df34-4ad7-87f7-62fb84ed915b None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:04:08.047057 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-a00cf304-df34-4ad7-87f7-62fb84ed915b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:04:08.110763 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7022c0-0a58-4e47-ab44-40583397b688 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:08.112691 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7022c0-0a58-4e47-ab44-40583397b688 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:08.156677 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 87/179] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:04:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 09:04:08.194229 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00cf304-df34-4ad7-87f7-62fb84ed915b None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:08.258249 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:08.268230 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 93/180] 10.222.0.26 () {58 vars in 1282 bytes} [Thu Jul 14 09:04:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 510 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 09:04:08.292153 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7022c0-0a58-4e47-ab44-40583397b688 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:08.373760 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00cf304-df34-4ad7-87f7-62fb84ed915b None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:08.404631 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:08.419054 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3b5188-55f2-446a-9147-992477b3b9b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:08.421543 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3b5188-55f2-446a-9147-992477b3b9b2 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:08.421978 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3b5188-55f2-446a-9147-992477b3b9b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:08.422394 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3b5188-55f2-446a-9147-992477b3b9b2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:08.433029 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7022c0-0a58-4e47-ab44-40583397b688 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:08.434926 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3b5188-55f2-446a-9147-992477b3b9b2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 09:04:08.443574 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-6b3b5188-55f2-446a-9147-992477b3b9b2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 09:04:08.443574 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:04:08.443574 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:04:08.443574 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:04:08.443574 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:04:08.443574 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:04:08.443574 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:04:08.443574 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:04:08.443574 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:04:08.443574 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:04:08.443574 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:04:08.443574 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:04:08.443574 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 09:04:08.443574 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 09:04:08.443574 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 09:04:08.443574 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 09:04:08.444676 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 09:04:08.444676 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 09:04:08.444676 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 09:04:08.444676 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 09:04:08.444676 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:04:08.444676 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:04:08.444676 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:04:08.444676 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:04:08.444676 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:04:08.444676 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:04:08.444676 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:04:08.444676 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 14 09:04:08.444676 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:04:08.444676 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:04:08.444676 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:04:08.444676 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:04:08.445689 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:04:08.445689 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:04:08.445689 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:04:08.445689 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:04:08.445689 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 09:04:08.445689 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 09:04:08.445689 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 09:04:08.445689 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 09:04:08.445689 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 09:04:08.445689 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 09:04:08.445689 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 09:04:08.445689 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 14 09:04:08.447957 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 88/181] 10.222.0.26 () {56 vars in 1096 bytes} [Thu Jul 14 09:04:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:04:08.449514 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185e12f0-ccf6-4357-bd3c-e6fa7af418bb None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:08.449943 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185e12f0-ccf6-4357-bd3c-e6fa7af418bb None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:08.450357 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185e12f0-ccf6-4357-bd3c-e6fa7af418bb None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:08.451910 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 89/182] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:04:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:04:08.453939 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185e12f0-ccf6-4357-bd3c-e6fa7af418bb None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:08.470318 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b57f3c2-9aa3-43fb-8d58-ddda10961e72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:08.471474 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b57f3c2-9aa3-43fb-8d58-ddda10961e72 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:08.471776 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b57f3c2-9aa3-43fb-8d58-ddda10961e72 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:08.472489 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b57f3c2-9aa3-43fb-8d58-ddda10961e72 None admin] PATH_INFO: `/v3/projects/4a4ff416931141ce83908e05cc434590` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:08.473996 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b57f3c2-9aa3-43fb-8d58-ddda10961e72 None admin] RBAC: Authorizing `identity:get_project(project_id=4a4ff416931141ce83908e05cc434590)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:08.475764 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b57f3c2-9aa3-43fb-8d58-ddda10961e72 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:08.484779 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 94/183] 10.222.0.26 () {56 vars in 1186 bytes} [Thu Jul 14 09:04:08 2022] GET /identity/v3/projects/4a4ff416931141ce83908e05cc434590 => generated 298 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 09:04:08.484779 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e388eff6-9f58-4383-8737-1ae300e4a146 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:08.484779 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e388eff6-9f58-4383-8737-1ae300e4a146 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:08.484779 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e388eff6-9f58-4383-8737-1ae300e4a146 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:08.497961 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-e388eff6-9f58-4383-8737-1ae300e4a146 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:04:08.499430 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-e388eff6-9f58-4383-8737-1ae300e4a146 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:04:08.574475 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:08.583351 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 95/184] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:04:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 09:04:08.584386 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e388eff6-9f58-4383-8737-1ae300e4a146 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:08.586025 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185e12f0-ccf6-4357-bd3c-e6fa7af418bb None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:08.621140 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee68c141-d906-4b84-aaf1-eec53de621d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:08.621805 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee68c141-d906-4b84-aaf1-eec53de621d4 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:08.622082 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee68c141-d906-4b84-aaf1-eec53de621d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:08.622350 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee68c141-d906-4b84-aaf1-eec53de621d4 None admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:08.623213 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee68c141-d906-4b84-aaf1-eec53de621d4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:04:08.623513 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee68c141-d906-4b84-aaf1-eec53de621d4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:08.625146 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee68c141-d906-4b84-aaf1-eec53de621d4 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:08.638352 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 90/185] 10.222.0.26 () {56 vars in 1099 bytes} [Thu Jul 14 09:04:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 09:04:08.639883 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b886301-6fdb-4185-a1bc-0a73fa4691bd None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:08.640165 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b886301-6fdb-4185-a1bc-0a73fa4691bd None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:08.640440 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b886301-6fdb-4185-a1bc-0a73fa4691bd None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:08.645675 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 91/186] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:04:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:04:08.645675 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b886301-6fdb-4185-a1bc-0a73fa4691bd None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:08.693812 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:08.694460 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e388eff6-9f58-4383-8737-1ae300e4a146 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:08.729138 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e43b16-5f50-4520-b3d9-7f5f625391a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:08.729935 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e43b16-5f50-4520-b3d9-7f5f625391a8 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:08.730220 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e43b16-5f50-4520-b3d9-7f5f625391a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:08.730499 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e43b16-5f50-4520-b3d9-7f5f625391a8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:08.731419 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e43b16-5f50-4520-b3d9-7f5f625391a8 None admin] RBAC: Adding query filter params (role.id=c4bab4a02e5c4f5f97a024e844a0c072, scope.project.id=4a4ff416931141ce83908e05cc434590, user.id=cb553a2f9c234bcbadbe3b9dbdd4a92e) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:04:08.731732 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e43b16-5f50-4520-b3d9-7f5f625391a8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:08.733201 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e43b16-5f50-4520-b3d9-7f5f625391a8 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:08.739159 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 96/187] 10.222.0.26 () {56 vars in 1374 bytes} [Thu Jul 14 09:04:08 2022] GET /identity/v3/role_assignments?user.id=cb553a2f9c234bcbadbe3b9dbdd4a92e&scope.project.id=4a4ff416931141ce83908e05cc434590&role.id=c4bab4a02e5c4f5f97a024e844a0c072 => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:04:08.739986 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e43b16-5f50-4520-b3d9-7f5f625391a8 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:08.768269 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b886301-6fdb-4185-a1bc-0a73fa4691bd None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:08.817417 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:08.825672 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b3a09d-672e-463e-b728-307c4651c5f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:08.827134 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b3a09d-672e-463e-b728-307c4651c5f7 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:08.827631 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b3a09d-672e-463e-b728-307c4651c5f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:08.828337 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b3a09d-672e-463e-b728-307c4651c5f7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:08.844532 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b3a09d-672e-463e-b728-307c4651c5f7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 09:04:08.856095 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-75b3a09d-672e-463e-b728-307c4651c5f7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 09:04:08.856095 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:04:08.856095 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:04:08.856095 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:04:08.856095 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:04:08.856095 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:04:08.856095 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:04:08.856095 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:04:08.856095 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:04:08.856095 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:04:08.856095 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:04:08.856095 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:04:08.856095 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 09:04:08.856095 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 09:04:08.856095 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 09:04:08.856095 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 09:04:08.857076 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 09:04:08.857076 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 09:04:08.857076 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 09:04:08.857076 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 09:04:08.857076 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:04:08.857076 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:04:08.857076 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:04:08.857076 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:04:08.857076 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:04:08.857076 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:04:08.857076 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:04:08.857076 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 14 09:04:08.857076 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:04:08.857076 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:04:08.857076 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:04:08.857076 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:04:08.857876 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:04:08.857876 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:04:08.857876 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:04:08.857876 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:04:08.857876 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 09:04:08.857876 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 09:04:08.857876 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 09:04:08.857876 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 09:04:08.857876 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 09:04:08.857876 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 09:04:08.857876 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 09:04:08.857876 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 14 09:04:08.859396 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 92/188] 10.222.0.26 () {56 vars in 1114 bytes} [Thu Jul 14 09:04:08 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:04:08.860943 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af67e022-9e97-4ff6-b92a-fbf685872075 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:08.861434 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af67e022-9e97-4ff6-b92a-fbf685872075 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:08.861906 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af67e022-9e97-4ff6-b92a-fbf685872075 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:08.870176 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e43b16-5f50-4520-b3d9-7f5f625391a8 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:08.885145 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-af67e022-9e97-4ff6-b92a-fbf685872075 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:04:08.887643 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-af67e022-9e97-4ff6-b92a-fbf685872075 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:04:08.911563 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12445a4f-4bef-44c4-a310-9ecd75e6fc59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:08.912887 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12445a4f-4bef-44c4-a310-9ecd75e6fc59 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:08.913510 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12445a4f-4bef-44c4-a310-9ecd75e6fc59 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:08.914099 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12445a4f-4bef-44c4-a310-9ecd75e6fc59 None admin] PATH_INFO: `/v3/users/00e19d5626ee486082115e1a5c2d2876` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:08.930779 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12445a4f-4bef-44c4-a310-9ecd75e6fc59 None admin] RBAC: Authorizing `identity:get_user(user_id=00e19d5626ee486082115e1a5c2d2876)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:08.933033 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12445a4f-4bef-44c4-a310-9ecd75e6fc59 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:08.934033 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 97/189] 10.222.0.26 () {56 vars in 1177 bytes} [Thu Jul 14 09:04:08 2022] GET /identity/v3/users/00e19d5626ee486082115e1a5c2d2876 => generated 283 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:04:08.935295 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12445a4f-4bef-44c4-a310-9ecd75e6fc59 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:08.974276 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 93/190] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:04:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 09:04:08.975599 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af67e022-9e97-4ff6-b92a-fbf685872075 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:09.016412 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:09.073436 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12445a4f-4bef-44c4-a310-9ecd75e6fc59 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:09.090089 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af67e022-9e97-4ff6-b92a-fbf685872075 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:09.095790 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:09.111516 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6a1531-83a3-4cfb-bb2b-c4ff2e8cce03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:09.112198 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6a1531-83a3-4cfb-bb2b-c4ff2e8cce03 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:09.112485 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6a1531-83a3-4cfb-bb2b-c4ff2e8cce03 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:09.112765 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6a1531-83a3-4cfb-bb2b-c4ff2e8cce03 None admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:09.113523 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6a1531-83a3-4cfb-bb2b-c4ff2e8cce03 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:04:09.113833 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6a1531-83a3-4cfb-bb2b-c4ff2e8cce03 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:09.115236 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6a1531-83a3-4cfb-bb2b-c4ff2e8cce03 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:09.123336 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 98/191] 10.222.0.26 () {56 vars in 1111 bytes} [Thu Jul 14 09:04:08 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:04:09.124129 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6a1531-83a3-4cfb-bb2b-c4ff2e8cce03 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:09.129540 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88328086-2797-43e0-a04d-1adb9ccc9d88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:09.130200 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88328086-2797-43e0-a04d-1adb9ccc9d88 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:09.130486 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88328086-2797-43e0-a04d-1adb9ccc9d88 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:09.130760 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88328086-2797-43e0-a04d-1adb9ccc9d88 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:09.203216 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88328086-2797-43e0-a04d-1adb9ccc9d88 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:09.204720 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88328086-2797-43e0-a04d-1adb9ccc9d88 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:09.232883 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6a1531-83a3-4cfb-bb2b-c4ff2e8cce03 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:09.240383 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:09.270962 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1014c26f-136d-47d3-a616-05067e3d1a24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:09.271806 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1014c26f-136d-47d3-a616-05067e3d1a24 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:09.272092 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1014c26f-136d-47d3-a616-05067e3d1a24 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:09.272373 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1014c26f-136d-47d3-a616-05067e3d1a24 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:09.273538 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1014c26f-136d-47d3-a616-05067e3d1a24 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:09.275065 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1014c26f-136d-47d3-a616-05067e3d1a24 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:09.276982 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 99/192] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jul 14 09:04:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:04:09.277575 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1014c26f-136d-47d3-a616-05067e3d1a24 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:09.310942 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 94/193] 10.222.0.26 () {58 vars in 1282 bytes} [Thu Jul 14 09:04:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 09:04:09.311645 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88328086-2797-43e0-a04d-1adb9ccc9d88 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:09.337229 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:09.389344 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1014c26f-136d-47d3-a616-05067e3d1a24 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:09.414186 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88328086-2797-43e0-a04d-1adb9ccc9d88 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:09.425256 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5831c00-da6b-4fb3-8dc6-4287ac9bc800 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:09.426201 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5831c00-da6b-4fb3-8dc6-4287ac9bc800 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:09.426485 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5831c00-da6b-4fb3-8dc6-4287ac9bc800 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:09.426762 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5831c00-da6b-4fb3-8dc6-4287ac9bc800 None admin] PATH_INFO: `/v3/users/00e19d5626ee486082115e1a5c2d2876` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:09.428392 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5831c00-da6b-4fb3-8dc6-4287ac9bc800 None admin] RBAC: Authorizing `identity:get_user(user_id=00e19d5626ee486082115e1a5c2d2876)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:09.430358 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5831c00-da6b-4fb3-8dc6-4287ac9bc800 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:09.436933 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 100/194] 10.222.0.26 () {56 vars in 1177 bytes} [Thu Jul 14 09:04:09 2022] GET /identity/v3/users/00e19d5626ee486082115e1a5c2d2876 => generated 283 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:04:09.436933 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5831c00-da6b-4fb3-8dc6-4287ac9bc800 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:09.452674 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eebb895-b988-4397-9c8a-8a6e32dfdf81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:09.453359 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eebb895-b988-4397-9c8a-8a6e32dfdf81 None admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:09.453644 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eebb895-b988-4397-9c8a-8a6e32dfdf81 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:09.453919 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eebb895-b988-4397-9c8a-8a6e32dfdf81 None admin] PATH_INFO: `/v3/users` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:09.454907 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eebb895-b988-4397-9c8a-8a6e32dfdf81 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:09.457041 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eebb895-b988-4397-9c8a-8a6e32dfdf81 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:09.465487 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:09.495230 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 95/195] 10.222.0.26 () {60 vars in 1132 bytes} [Thu Jul 14 09:04:09 2022] POST /identity/v3/users => generated 297 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 09:04:09.495230 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eebb895-b988-4397-9c8a-8a6e32dfdf81 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:09.560568 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5831c00-da6b-4fb3-8dc6-4287ac9bc800 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:09.583628 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:09.604532 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f08b38-5fbb-4877-a459-76f8e48296cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:09.605858 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f08b38-5fbb-4877-a459-76f8e48296cb None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:09.605858 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f08b38-5fbb-4877-a459-76f8e48296cb None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:09.606033 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f08b38-5fbb-4877-a459-76f8e48296cb None admin] PATH_INFO: `/v3/projects/a596189576264bfc9638ca33f184759c` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:09.607343 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f08b38-5fbb-4877-a459-76f8e48296cb None admin] RBAC: Authorizing `identity:get_project(project_id=a596189576264bfc9638ca33f184759c)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:09.608961 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f08b38-5fbb-4877-a459-76f8e48296cb None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:09.611015 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 101/196] 10.222.0.26 () {56 vars in 1186 bytes} [Thu Jul 14 09:04:09 2022] GET /identity/v3/projects/a596189576264bfc9638ca33f184759c => generated 302 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:04:09.617263 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f08b38-5fbb-4877-a459-76f8e48296cb None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:09.640353 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eebb895-b988-4397-9c8a-8a6e32dfdf81 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:09.684425 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3900765a-dae6-4eb7-9b45-425ef624ef1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:09.685469 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3900765a-dae6-4eb7-9b45-425ef624ef1e None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:09.685955 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3900765a-dae6-4eb7-9b45-425ef624ef1e None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:09.686431 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3900765a-dae6-4eb7-9b45-425ef624ef1e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:09.734525 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f08b38-5fbb-4877-a459-76f8e48296cb None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:09.753419 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:09.771304 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3900765a-dae6-4eb7-9b45-425ef624ef1e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:09.772964 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3900765a-dae6-4eb7-9b45-425ef624ef1e None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:09.784074 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741fe25a-a874-440c-81a2-7eaffd4c412d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:09.784755 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741fe25a-a874-440c-81a2-7eaffd4c412d None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:09.785036 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741fe25a-a874-440c-81a2-7eaffd4c412d None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:09.785308 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741fe25a-a874-440c-81a2-7eaffd4c412d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:09.786209 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741fe25a-a874-440c-81a2-7eaffd4c412d None admin] RBAC: Adding query filter params (role.id=7b549d58d1724395b343609d895c80d0, scope.project.id=a596189576264bfc9638ca33f184759c, user.id=00e19d5626ee486082115e1a5c2d2876) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:04:09.786526 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741fe25a-a874-440c-81a2-7eaffd4c412d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:09.788020 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741fe25a-a874-440c-81a2-7eaffd4c412d None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:09.794809 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 102/197] 10.222.0.26 () {56 vars in 1374 bytes} [Thu Jul 14 09:04:09 2022] GET /identity/v3/role_assignments?user.id=00e19d5626ee486082115e1a5c2d2876&scope.project.id=a596189576264bfc9638ca33f184759c&role.id=7b549d58d1724395b343609d895c80d0 => generated 259 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 09:04:09.899491 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 96/198] 10.222.0.26 () {58 vars in 1282 bytes} [Thu Jul 14 09:04:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 09:04:09.904996 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741fe25a-a874-440c-81a2-7eaffd4c412d None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:09.972522 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:10.030104 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741fe25a-a874-440c-81a2-7eaffd4c412d None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:10.070627 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d288e3c-06c7-4c3b-8b02-34fdbc4bd997 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:10.071580 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d288e3c-06c7-4c3b-8b02-34fdbc4bd997 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:10.071861 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d288e3c-06c7-4c3b-8b02-34fdbc4bd997 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:10.072134 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d288e3c-06c7-4c3b-8b02-34fdbc4bd997 None admin] PATH_INFO: `/v3/projects/a596189576264bfc9638ca33f184759c` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:10.073421 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d288e3c-06c7-4c3b-8b02-34fdbc4bd997 None admin] RBAC: Authorizing `identity:get_project(project_id=a596189576264bfc9638ca33f184759c)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:10.074885 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d288e3c-06c7-4c3b-8b02-34fdbc4bd997 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:10.077045 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 103/199] 10.222.0.26 () {56 vars in 1186 bytes} [Thu Jul 14 09:04:09 2022] GET /identity/v3/projects/a596189576264bfc9638ca33f184759c => generated 302 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:04:10.082662 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3900765a-dae6-4eb7-9b45-425ef624ef1e None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:10.088391 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13dddaf-17f2-4f33-b2cc-0a463e47b32d None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:10.088677 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13dddaf-17f2-4f33-b2cc-0a463e47b32d None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:10.088956 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13dddaf-17f2-4f33-b2cc-0a463e47b32d None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:10.090584 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 104/200] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:04:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:04:10.096261 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfa2589-b208-4d2e-9dd6-d2815ac8cff0 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:10.096261 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfa2589-b208-4d2e-9dd6-d2815ac8cff0 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:10.096261 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfa2589-b208-4d2e-9dd6-d2815ac8cff0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:10.113022 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-fcfa2589-b208-4d2e-9dd6-d2815ac8cff0 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:04:10.114597 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-fcfa2589-b208-4d2e-9dd6-d2815ac8cff0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:04:10.187035 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 105/201] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:04:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 09:04:10.209379 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfa2589-b208-4d2e-9dd6-d2815ac8cff0 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:10.267087 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:10.308107 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:10.319913 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3900765a-dae6-4eb7-9b45-425ef624ef1e None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:10.363750 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521512a8-b852-4dcb-99f4-e8d9099166bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:10.365185 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521512a8-b852-4dcb-99f4-e8d9099166bc None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:10.365808 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521512a8-b852-4dcb-99f4-e8d9099166bc None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:10.366260 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521512a8-b852-4dcb-99f4-e8d9099166bc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:10.367314 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521512a8-b852-4dcb-99f4-e8d9099166bc None admin] RBAC: Adding query filter params (role.id=c4bab4a02e5c4f5f97a024e844a0c072, scope.project.id=a596189576264bfc9638ca33f184759c, user.id=00e19d5626ee486082115e1a5c2d2876) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:04:10.367700 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfa2589-b208-4d2e-9dd6-d2815ac8cff0 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:10.368051 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521512a8-b852-4dcb-99f4-e8d9099166bc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:10.369948 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521512a8-b852-4dcb-99f4-e8d9099166bc None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:10.376762 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 97/202] 10.222.0.26 () {56 vars in 1374 bytes} [Thu Jul 14 09:04:10 2022] GET /identity/v3/role_assignments?user.id=00e19d5626ee486082115e1a5c2d2876&scope.project.id=a596189576264bfc9638ca33f184759c&role.id=c4bab4a02e5c4f5f97a024e844a0c072 => generated 259 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:04:10.418725 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72842404-4e93-46c7-9a8b-51948cec861c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:10.419600 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72842404-4e93-46c7-9a8b-51948cec861c None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:10.419883 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72842404-4e93-46c7-9a8b-51948cec861c None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:10.420162 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72842404-4e93-46c7-9a8b-51948cec861c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:10.441345 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72842404-4e93-46c7-9a8b-51948cec861c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 09:04:10.449454 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-72842404-4e93-46c7-9a8b-51948cec861c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 09:04:10.449454 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:04:10.449454 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:04:10.449454 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:04:10.449454 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:04:10.449454 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:04:10.449454 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:04:10.449454 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:04:10.449454 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:04:10.449454 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:04:10.449454 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:04:10.449454 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:04:10.449454 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 09:04:10.449454 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 09:04:10.449454 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 09:04:10.449454 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 09:04:10.450422 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 09:04:10.450422 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 09:04:10.450422 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 09:04:10.450422 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 09:04:10.450422 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:04:10.450422 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:04:10.450422 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:04:10.450422 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:04:10.450422 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:04:10.450422 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:04:10.450422 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:04:10.450422 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 14 09:04:10.450422 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:04:10.450422 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:04:10.450422 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:04:10.450422 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:04:10.451281 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:04:10.451281 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:04:10.451281 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:04:10.451281 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:04:10.451281 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 09:04:10.451281 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 09:04:10.451281 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 09:04:10.451281 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 09:04:10.451281 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 09:04:10.451281 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 09:04:10.451281 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 09:04:10.451281 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 14 09:04:10.451281 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 106/203] 10.222.0.26 () {56 vars in 1099 bytes} [Thu Jul 14 09:04:10 2022] GET /identity/v3/roles/reader => generated 84 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:04:10.456961 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521512a8-b852-4dcb-99f4-e8d9099166bc None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:10.621394 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521512a8-b852-4dcb-99f4-e8d9099166bc None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:10.666254 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9851c53-0908-4f67-a671-6a084aa62a4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:10.667031 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9851c53-0908-4f67-a671-6a084aa62a4b None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:10.667227 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9851c53-0908-4f67-a671-6a084aa62a4b None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:10.667508 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9851c53-0908-4f67-a671-6a084aa62a4b None admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:10.668271 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9851c53-0908-4f67-a671-6a084aa62a4b None admin] RBAC: Adding query filter params (name=reader) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:04:10.668577 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9851c53-0908-4f67-a671-6a084aa62a4b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:10.669959 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9851c53-0908-4f67-a671-6a084aa62a4b None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:10.683085 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 98/204] 10.222.0.26 () {56 vars in 1101 bytes} [Thu Jul 14 09:04:10 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:04:10.688853 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72842404-4e93-46c7-9a8b-51948cec861c None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:10.793039 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72842404-4e93-46c7-9a8b-51948cec861c None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:10.802553 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:10.840822 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6af4a6-65fa-4985-831f-ea349fcd2086 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:10.841707 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6af4a6-65fa-4985-831f-ea349fcd2086 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:10.841982 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6af4a6-65fa-4985-831f-ea349fcd2086 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:10.842248 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6af4a6-65fa-4985-831f-ea349fcd2086 None admin] PATH_INFO: `/v3/users/f27c191891e34eeda402e573588c394c` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:10.847907 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6af4a6-65fa-4985-831f-ea349fcd2086 None admin] RBAC: Authorizing `identity:get_user(user_id=f27c191891e34eeda402e573588c394c)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:10.849385 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6af4a6-65fa-4985-831f-ea349fcd2086 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:10.851229 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 107/205] 10.222.0.26 () {56 vars in 1177 bytes} [Thu Jul 14 09:04:10 2022] GET /identity/v3/users/f27c191891e34eeda402e573588c394c => generated 289 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:04:10.856541 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9851c53-0908-4f67-a671-6a084aa62a4b None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:10.890896 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:11.022842 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9851c53-0908-4f67-a671-6a084aa62a4b None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:11.068813 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02a3a0d-2577-4f61-89fa-6b3c3062f484 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:11.069392 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02a3a0d-2577-4f61-89fa-6b3c3062f484 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:11.069683 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02a3a0d-2577-4f61-89fa-6b3c3062f484 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:11.069962 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02a3a0d-2577-4f61-89fa-6b3c3062f484 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:11.162574 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02a3a0d-2577-4f61-89fa-6b3c3062f484 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:11.164104 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02a3a0d-2577-4f61-89fa-6b3c3062f484 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:11.192248 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:11.307002 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 99/206] 10.222.0.26 () {58 vars in 1282 bytes} [Thu Jul 14 09:04:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 09:04:11.312142 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6af4a6-65fa-4985-831f-ea349fcd2086 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:11.474099 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6af4a6-65fa-4985-831f-ea349fcd2086 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:11.514673 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224b3730-0540-4dcb-be27-74356ead0f70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:11.515953 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224b3730-0540-4dcb-be27-74356ead0f70 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:11.516434 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224b3730-0540-4dcb-be27-74356ead0f70 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:11.516899 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224b3730-0540-4dcb-be27-74356ead0f70 None admin] PATH_INFO: `/v3/projects/4a4ff416931141ce83908e05cc434590` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:11.518332 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224b3730-0540-4dcb-be27-74356ead0f70 None admin] RBAC: Authorizing `identity:get_project(project_id=4a4ff416931141ce83908e05cc434590)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:11.520059 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224b3730-0540-4dcb-be27-74356ead0f70 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:11.522325 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 108/207] 10.222.0.26 () {56 vars in 1186 bytes} [Thu Jul 14 09:04:11 2022] GET /identity/v3/projects/4a4ff416931141ce83908e05cc434590 => generated 298 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:04:11.527945 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02a3a0d-2577-4f61-89fa-6b3c3062f484 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:11.698967 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02a3a0d-2577-4f61-89fa-6b3c3062f484 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:11.722459 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:11.747976 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69d4a3c-89eb-408c-aa3d-dab786283a71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:11.748559 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69d4a3c-89eb-408c-aa3d-dab786283a71 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:11.748843 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69d4a3c-89eb-408c-aa3d-dab786283a71 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:11.749114 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69d4a3c-89eb-408c-aa3d-dab786283a71 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:11.749960 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69d4a3c-89eb-408c-aa3d-dab786283a71 None admin] RBAC: Adding query filter params (role.id=9c412baac5914ce6a40ad7d3f0f0a0d6, scope.project.id=4a4ff416931141ce83908e05cc434590, user.id=f27c191891e34eeda402e573588c394c) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:04:11.750267 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69d4a3c-89eb-408c-aa3d-dab786283a71 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:11.751749 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69d4a3c-89eb-408c-aa3d-dab786283a71 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:11.757913 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 100/208] 10.222.0.26 () {56 vars in 1374 bytes} [Thu Jul 14 09:04:11 2022] GET /identity/v3/role_assignments?user.id=f27c191891e34eeda402e573588c394c&scope.project.id=4a4ff416931141ce83908e05cc434590&role.id=9c412baac5914ce6a40ad7d3f0f0a0d6 => generated 608 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:04:12.077675 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b07ffb-ea28-4006-a8da-55d94e4856db None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:12.078646 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b07ffb-ea28-4006-a8da-55d94e4856db None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:12.079145 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b07ffb-ea28-4006-a8da-55d94e4856db None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:12.081024 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 109/209] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:04:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:04:12.089267 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98fb97b-b057-4f08-bcb9-46189e99cb3d None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:12.089551 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98fb97b-b057-4f08-bcb9-46189e99cb3d None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:12.089834 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98fb97b-b057-4f08-bcb9-46189e99cb3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:12.115567 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-c98fb97b-b057-4f08-bcb9-46189e99cb3d None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:04:12.116755 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-c98fb97b-b057-4f08-bcb9-46189e99cb3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:04:12.187117 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0256873-29c0-4a50-8923-e3e272d06f94 None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:12.187451 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0256873-29c0-4a50-8923-e3e272d06f94 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:12.187890 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0256873-29c0-4a50-8923-e3e272d06f94 None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:12.189292 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 110/210] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:04:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:04:12.196014 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb7b152-fa5e-40b5-8353-43a91d4d32c2 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:12.196611 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb7b152-fa5e-40b5-8353-43a91d4d32c2 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:12.197115 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb7b152-fa5e-40b5-8353-43a91d4d32c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:12.200079 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 101/211] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:04:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 09:04:12.214620 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-bcb7b152-fa5e-40b5-8353-43a91d4d32c2 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:04:12.216143 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-bcb7b152-fa5e-40b5-8353-43a91d4d32c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:04:12.227035 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98fb97b-b057-4f08-bcb9-46189e99cb3d None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:12.276975 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:12.303129 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 111/212] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:04:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 09:04:12.309124 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137e62b6-b353-443b-b534-628d4368d723 None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:12.309662 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137e62b6-b353-443b-b534-628d4368d723 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:12.309949 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137e62b6-b353-443b-b534-628d4368d723 None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:12.315309 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 112/213] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:04:12 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:04:12.316637 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7534ed5c-cd54-49f3-9ac6-4b6ccc8c9642 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:12.318795 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7534ed5c-cd54-49f3-9ac6-4b6ccc8c9642 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:12.319311 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7534ed5c-cd54-49f3-9ac6-4b6ccc8c9642 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:12.337180 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-7534ed5c-cd54-49f3-9ac6-4b6ccc8c9642 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:04:12.338274 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-7534ed5c-cd54-49f3-9ac6-4b6ccc8c9642 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:04:12.347932 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98fb97b-b057-4f08-bcb9-46189e99cb3d None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:12.359813 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:12.384938 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292a78cc-4394-4b49-a26a-75d9082ec6f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:12.385705 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292a78cc-4394-4b49-a26a-75d9082ec6f8 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:12.385977 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292a78cc-4394-4b49-a26a-75d9082ec6f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:12.386454 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292a78cc-4394-4b49-a26a-75d9082ec6f8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:12.396677 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292a78cc-4394-4b49-a26a-75d9082ec6f8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 09:04:12.404901 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-292a78cc-4394-4b49-a26a-75d9082ec6f8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 09:04:12.404901 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:04:12.404901 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:04:12.404901 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:04:12.404901 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:04:12.404901 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:04:12.404901 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:04:12.404901 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:04:12.404901 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:04:12.404901 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:04:12.404901 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:04:12.404901 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:04:12.404901 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 09:04:12.404901 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 09:04:12.404901 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 09:04:12.404901 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 09:04:12.405777 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 09:04:12.405777 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 09:04:12.405777 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 09:04:12.405777 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 09:04:12.405777 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:04:12.405777 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:04:12.405777 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:04:12.405777 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:04:12.405777 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:04:12.405777 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:04:12.405777 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:04:12.405777 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 14 09:04:12.405777 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:04:12.405777 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:04:12.405777 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:04:12.405777 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:04:12.413526 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:04:12.413526 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:04:12.413526 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:04:12.413526 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:04:12.413526 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 09:04:12.413526 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 09:04:12.413526 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 09:04:12.413526 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 09:04:12.413526 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 09:04:12.413526 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 09:04:12.413526 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 09:04:12.413526 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 14 09:04:12.414708 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 113/214] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:04:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 09:04:12.414832 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 102/215] 10.222.0.26 () {56 vars in 1099 bytes} [Thu Jul 14 09:04:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 09:04:12.414990 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7534ed5c-cd54-49f3-9ac6-4b6ccc8c9642 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:12.415135 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292a78cc-4394-4b49-a26a-75d9082ec6f8 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:12.528587 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7534ed5c-cd54-49f3-9ac6-4b6ccc8c9642 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:12.533902 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:12.535427 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292a78cc-4394-4b49-a26a-75d9082ec6f8 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:12.568385 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d718890-05b7-49ad-836e-9b27a59ed808 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:12.569348 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d718890-05b7-49ad-836e-9b27a59ed808 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:12.569634 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d718890-05b7-49ad-836e-9b27a59ed808 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:12.570215 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d718890-05b7-49ad-836e-9b27a59ed808 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:12.577822 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f4963e-1c00-48c4-8f91-8d1ee76b6f01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:12.578499 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f4963e-1c00-48c4-8f91-8d1ee76b6f01 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:12.578783 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f4963e-1c00-48c4-8f91-8d1ee76b6f01 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:12.579068 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f4963e-1c00-48c4-8f91-8d1ee76b6f01 None admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:12.579854 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f4963e-1c00-48c4-8f91-8d1ee76b6f01 None admin] RBAC: Adding query filter params (name=member) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:04:12.580165 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f4963e-1c00-48c4-8f91-8d1ee76b6f01 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:12.581592 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f4963e-1c00-48c4-8f91-8d1ee76b6f01 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:12.581824 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d718890-05b7-49ad-836e-9b27a59ed808 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 09:04:12.590172 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 103/216] 10.222.0.26 () {56 vars in 1101 bytes} [Thu Jul 14 09:04:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 09:04:12.590843 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f4963e-1c00-48c4-8f91-8d1ee76b6f01 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:12.594320 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-0d718890-05b7-49ad-836e-9b27a59ed808 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 09:04:12.594320 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:04:12.594320 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:04:12.594320 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:04:12.594320 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:04:12.594320 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:04:12.594320 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:04:12.594320 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:04:12.594320 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:04:12.594320 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:04:12.594320 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:04:12.594320 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:04:12.594320 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 09:04:12.594320 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 09:04:12.594320 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 09:04:12.594320 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 09:04:12.595515 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 09:04:12.595515 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 09:04:12.595515 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 09:04:12.595515 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 09:04:12.595515 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:04:12.595515 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:04:12.595515 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:04:12.595515 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:04:12.595515 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:04:12.595515 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:04:12.595515 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:04:12.595515 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 14 09:04:12.595515 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:04:12.595515 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:04:12.595515 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:04:12.595515 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:04:12.596472 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:04:12.596472 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:04:12.596472 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:04:12.596472 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:04:12.596472 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 09:04:12.596472 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 09:04:12.596472 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 09:04:12.596472 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 09:04:12.596472 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 09:04:12.596472 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 09:04:12.596472 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 09:04:12.596472 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 14 09:04:12.596472 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 114/217] 10.222.0.26 () {56 vars in 1096 bytes} [Thu Jul 14 09:04:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:04:12.597218 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d718890-05b7-49ad-836e-9b27a59ed808 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:12.629953 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:12.699883 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d718890-05b7-49ad-836e-9b27a59ed808 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:12.700486 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f4963e-1c00-48c4-8f91-8d1ee76b6f01 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:12.734205 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af55a2d-4aff-4a4b-b818-420f4501cd19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:12.735086 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af55a2d-4aff-4a4b-b818-420f4501cd19 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:12.735399 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af55a2d-4aff-4a4b-b818-420f4501cd19 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:12.735494 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002ced5f-2a65-434b-b461-c899fe66a734 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:12.735671 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af55a2d-4aff-4a4b-b818-420f4501cd19 None admin] PATH_INFO: `/v3/users/06a6d5cbc4324313a9f8745928cd5ca8` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:12.736345 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002ced5f-2a65-434b-b461-c899fe66a734 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:12.736710 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002ced5f-2a65-434b-b461-c899fe66a734 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:12.736985 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002ced5f-2a65-434b-b461-c899fe66a734 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:12.738149 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002ced5f-2a65-434b-b461-c899fe66a734 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:12.739673 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002ced5f-2a65-434b-b461-c899fe66a734 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:12.746319 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 104/218] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jul 14 09:04:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:04:12.746319 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002ced5f-2a65-434b-b461-c899fe66a734 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:12.752191 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af55a2d-4aff-4a4b-b818-420f4501cd19 None admin] RBAC: Authorizing `identity:get_user(user_id=06a6d5cbc4324313a9f8745928cd5ca8)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:12.753702 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af55a2d-4aff-4a4b-b818-420f4501cd19 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:12.755514 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 115/219] 10.222.0.26 () {56 vars in 1177 bytes} [Thu Jul 14 09:04:12 2022] GET /identity/v3/users/06a6d5cbc4324313a9f8745928cd5ca8 => generated 297 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:04:12.756260 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af55a2d-4aff-4a4b-b818-420f4501cd19 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:12.780130 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:12.853539 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:12.856901 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002ced5f-2a65-434b-b461-c899fe66a734 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:12.860202 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af55a2d-4aff-4a4b-b818-420f4501cd19 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:12.893733 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cb04fd-a1c9-421c-95c7-e600372ce6b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:12.894426 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cb04fd-a1c9-421c-95c7-e600372ce6b2 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:12.894711 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cb04fd-a1c9-421c-95c7-e600372ce6b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:12.895153 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cb04fd-a1c9-421c-95c7-e600372ce6b2 None admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:12.895755 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cb04fd-a1c9-421c-95c7-e600372ce6b2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:04:12.896056 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cb04fd-a1c9-421c-95c7-e600372ce6b2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:12.897441 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cb04fd-a1c9-421c-95c7-e600372ce6b2 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:12.900100 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0af3225-6db8-448f-82ba-f6af6767cd5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:12.900780 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0af3225-6db8-448f-82ba-f6af6767cd5b None admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:12.901061 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0af3225-6db8-448f-82ba-f6af6767cd5b None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:12.901333 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0af3225-6db8-448f-82ba-f6af6767cd5b None admin] PATH_INFO: `/v3/users` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:12.902051 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0af3225-6db8-448f-82ba-f6af6767cd5b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:12.903546 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0af3225-6db8-448f-82ba-f6af6767cd5b None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:12.906269 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 105/220] 10.222.0.26 () {56 vars in 1099 bytes} [Thu Jul 14 09:04:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:04:12.906861 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cb04fd-a1c9-421c-95c7-e600372ce6b2 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:12.929451 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 116/221] 10.222.0.26 () {60 vars in 1132 bytes} [Thu Jul 14 09:04:12 2022] POST /identity/v3/users => generated 297 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 09:04:12.930246 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0af3225-6db8-448f-82ba-f6af6767cd5b None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:13.017669 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:13.037497 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cb04fd-a1c9-421c-95c7-e600372ce6b2 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:13.043989 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0af3225-6db8-448f-82ba-f6af6767cd5b None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:13.075247 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684f81ec-d5d1-4705-9291-6c525f0fdf67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:13.075920 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684f81ec-d5d1-4705-9291-6c525f0fdf67 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:13.076191 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684f81ec-d5d1-4705-9291-6c525f0fdf67 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:13.076470 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684f81ec-d5d1-4705-9291-6c525f0fdf67 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:13.081182 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbf8c86-e576-40ea-bd43-002af77eebd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:13.082151 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbf8c86-e576-40ea-bd43-002af77eebd0 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:13.082434 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbf8c86-e576-40ea-bd43-002af77eebd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:13.082707 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbf8c86-e576-40ea-bd43-002af77eebd0 None admin] PATH_INFO: `/v3/users/00e19d5626ee486082115e1a5c2d2876` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:13.083928 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbf8c86-e576-40ea-bd43-002af77eebd0 None admin] RBAC: Authorizing `identity:get_user(user_id=00e19d5626ee486082115e1a5c2d2876)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:13.085396 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbf8c86-e576-40ea-bd43-002af77eebd0 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:13.087253 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 117/222] 10.222.0.26 () {56 vars in 1177 bytes} [Thu Jul 14 09:04:12 2022] GET /identity/v3/users/00e19d5626ee486082115e1a5c2d2876 => generated 283 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:04:13.092842 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762ec59b-4eda-4023-995d-211113252607 None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:13.093131 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762ec59b-4eda-4023-995d-211113252607 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:13.093407 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762ec59b-4eda-4023-995d-211113252607 None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:13.094751 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 118/223] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:04:13 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:04:13.095993 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762ec59b-4eda-4023-995d-211113252607 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:13.114193 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:13.156234 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684f81ec-d5d1-4705-9291-6c525f0fdf67 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:13.157714 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684f81ec-d5d1-4705-9291-6c525f0fdf67 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:13.207424 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762ec59b-4eda-4023-995d-211113252607 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:13.244922 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4a58af-0935-4ec6-a5b8-11ed27c8ad45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:13.245793 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4a58af-0935-4ec6-a5b8-11ed27c8ad45 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:13.246308 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4a58af-0935-4ec6-a5b8-11ed27c8ad45 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:13.246427 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4a58af-0935-4ec6-a5b8-11ed27c8ad45 None admin] PATH_INFO: `/v3/projects/a596189576264bfc9638ca33f184759c` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:13.247694 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4a58af-0935-4ec6-a5b8-11ed27c8ad45 None admin] RBAC: Authorizing `identity:get_project(project_id=a596189576264bfc9638ca33f184759c)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:13.249292 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4a58af-0935-4ec6-a5b8-11ed27c8ad45 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:13.250843 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 119/224] 10.222.0.26 () {56 vars in 1186 bytes} [Thu Jul 14 09:04:13 2022] GET /identity/v3/projects/a596189576264bfc9638ca33f184759c => generated 302 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:04:13.252048 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26544a0b-3962-46dd-b8cc-9311063a0ac2 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:13.252333 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26544a0b-3962-46dd-b8cc-9311063a0ac2 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:13.252554 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26544a0b-3962-46dd-b8cc-9311063a0ac2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:13.269083 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-26544a0b-3962-46dd-b8cc-9311063a0ac2 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:04:13.270276 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-26544a0b-3962-46dd-b8cc-9311063a0ac2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:04:13.278585 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 106/225] 10.222.0.26 () {58 vars in 1282 bytes} [Thu Jul 14 09:04:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 09:04:13.279909 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684f81ec-d5d1-4705-9291-6c525f0fdf67 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:13.280712 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:13.348459 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:13.358995 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 120/226] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:04:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 09:04:13.360332 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26544a0b-3962-46dd-b8cc-9311063a0ac2 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:13.412619 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684f81ec-d5d1-4705-9291-6c525f0fdf67 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:13.449990 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08a82b5-ec8c-4526-ac92-f8abc5bcadae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:13.450899 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08a82b5-ec8c-4526-ac92-f8abc5bcadae None admin] REQUEST_METHOD: `PUT` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:13.451203 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08a82b5-ec8c-4526-ac92-f8abc5bcadae None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:13.451492 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08a82b5-ec8c-4526-ac92-f8abc5bcadae None admin] PATH_INFO: `/v3/projects/a596189576264bfc9638ca33f184759c/users/00e19d5626ee486082115e1a5c2d2876/roles/7b549d58d1724395b343609d895c80d0` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:13.453500 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08a82b5-ec8c-4526-ac92-f8abc5bcadae None admin] RBAC: Authorizing `identity:create_grant(project_id=a596189576264bfc9638ca33f184759c, user_id=00e19d5626ee486082115e1a5c2d2876, role_id=7b549d58d1724395b343609d895c80d0)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:13.455083 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08a82b5-ec8c-4526-ac92-f8abc5bcadae None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:13.473448 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 107/227] 10.222.0.26 () {58 vars in 1439 bytes} [Thu Jul 14 09:04:13 2022] PUT /identity/v3/projects/a596189576264bfc9638ca33f184759c/users/00e19d5626ee486082115e1a5c2d2876/roles/7b549d58d1724395b343609d895c80d0 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:04:13.479507 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08a82b5-ec8c-4526-ac92-f8abc5bcadae None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:13.487383 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26544a0b-3962-46dd-b8cc-9311063a0ac2 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:13.561546 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b483d6-4845-4cd2-810a-d3bea459479b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:13.562870 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b483d6-4845-4cd2-810a-d3bea459479b None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:13.563389 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b483d6-4845-4cd2-810a-d3bea459479b None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:13.563888 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b483d6-4845-4cd2-810a-d3bea459479b None admin] PATH_INFO: `/v3/projects/a596189576264bfc9638ca33f184759c` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:13.568017 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b483d6-4845-4cd2-810a-d3bea459479b None admin] RBAC: Authorizing `identity:get_project(project_id=a596189576264bfc9638ca33f184759c)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:13.569825 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b483d6-4845-4cd2-810a-d3bea459479b None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:13.572338 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 121/228] 10.222.0.26 () {56 vars in 1186 bytes} [Thu Jul 14 09:04:13 2022] GET /identity/v3/projects/a596189576264bfc9638ca33f184759c => generated 302 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:04:13.578233 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b483d6-4845-4cd2-810a-d3bea459479b None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:13.599958 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08a82b5-ec8c-4526-ac92-f8abc5bcadae None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:13.616100 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:13.661336 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:13.687059 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3434397-7223-435b-8326-93f33bbce7e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:13.687836 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3434397-7223-435b-8326-93f33bbce7e5 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:13.688120 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3434397-7223-435b-8326-93f33bbce7e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:13.688401 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3434397-7223-435b-8326-93f33bbce7e5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:13.701734 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3434397-7223-435b-8326-93f33bbce7e5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 09:04:13.711131 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-e3434397-7223-435b-8326-93f33bbce7e5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 09:04:13.711131 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:04:13.711131 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:04:13.711131 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:04:13.711131 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:04:13.711131 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:04:13.711131 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:04:13.711131 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:04:13.711131 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:04:13.711131 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:04:13.711131 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:04:13.711131 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:04:13.711131 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 09:04:13.711131 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 09:04:13.711131 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 09:04:13.711131 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 09:04:13.712055 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 09:04:13.712055 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 09:04:13.712055 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 09:04:13.712055 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 09:04:13.712055 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:04:13.712055 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:04:13.712055 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:04:13.712055 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:04:13.712055 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:04:13.712055 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:04:13.712055 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:04:13.712055 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 14 09:04:13.712055 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:04:13.712055 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:04:13.712055 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:04:13.712055 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:04:13.712834 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:04:13.712834 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:04:13.712834 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:04:13.712834 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:04:13.712834 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 09:04:13.712834 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 09:04:13.712834 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 09:04:13.712834 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 09:04:13.712834 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 09:04:13.712834 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 09:04:13.712834 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 09:04:13.712834 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 14 09:04:13.712834 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 108/229] 10.222.0.26 () {56 vars in 1114 bytes} [Thu Jul 14 09:04:13 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:04:13.718312 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3434397-7223-435b-8326-93f33bbce7e5 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:13.767994 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b483d6-4845-4cd2-810a-d3bea459479b None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:13.805518 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bab908-14e1-4c82-a2ee-f5cfc29dd06e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:13.806638 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bab908-14e1-4c82-a2ee-f5cfc29dd06e None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:13.807114 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bab908-14e1-4c82-a2ee-f5cfc29dd06e None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:13.807633 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bab908-14e1-4c82-a2ee-f5cfc29dd06e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:13.808663 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bab908-14e1-4c82-a2ee-f5cfc29dd06e None admin] RBAC: Adding query filter params (role.id=d1691322ce6a4edab7ce722edf0f2988, scope.project.id=a596189576264bfc9638ca33f184759c, user.id=06a6d5cbc4324313a9f8745928cd5ca8) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:04:13.809277 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bab908-14e1-4c82-a2ee-f5cfc29dd06e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:13.811039 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bab908-14e1-4c82-a2ee-f5cfc29dd06e None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:13.823623 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 122/230] 10.222.0.26 () {56 vars in 1374 bytes} [Thu Jul 14 09:04:13 2022] GET /identity/v3/role_assignments?user.id=06a6d5cbc4324313a9f8745928cd5ca8&scope.project.id=a596189576264bfc9638ca33f184759c&role.id=d1691322ce6a4edab7ce722edf0f2988 => generated 259 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:04:13.845481 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3434397-7223-435b-8326-93f33bbce7e5 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:13.896300 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c5976b-bc28-464e-8f9e-3556d4394e24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:13.897017 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c5976b-bc28-464e-8f9e-3556d4394e24 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:13.897308 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c5976b-bc28-464e-8f9e-3556d4394e24 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:13.897584 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c5976b-bc28-464e-8f9e-3556d4394e24 None admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:13.898444 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c5976b-bc28-464e-8f9e-3556d4394e24 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:04:13.898800 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c5976b-bc28-464e-8f9e-3556d4394e24 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:13.900290 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c5976b-bc28-464e-8f9e-3556d4394e24 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:13.909718 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 109/231] 10.222.0.26 () {56 vars in 1111 bytes} [Thu Jul 14 09:04:13 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:04:13.915510 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bab908-14e1-4c82-a2ee-f5cfc29dd06e None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:14.034982 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:14.059219 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bab908-14e1-4c82-a2ee-f5cfc29dd06e None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:14.094285 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd277e1b-2ccd-40fe-8e84-8b3a15b4ae5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:14.095179 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd277e1b-2ccd-40fe-8e84-8b3a15b4ae5e None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:14.095491 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd277e1b-2ccd-40fe-8e84-8b3a15b4ae5e None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:14.095767 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd277e1b-2ccd-40fe-8e84-8b3a15b4ae5e None admin] PATH_INFO: `/v3/users/00e19d5626ee486082115e1a5c2d2876` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:14.096973 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd277e1b-2ccd-40fe-8e84-8b3a15b4ae5e None admin] RBAC: Authorizing `identity:get_user(user_id=00e19d5626ee486082115e1a5c2d2876)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:14.098498 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd277e1b-2ccd-40fe-8e84-8b3a15b4ae5e None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:14.100538 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 123/232] 10.222.0.26 () {56 vars in 1177 bytes} [Thu Jul 14 09:04:13 2022] GET /identity/v3/users/00e19d5626ee486082115e1a5c2d2876 => generated 283 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:04:14.106270 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c5976b-bc28-464e-8f9e-3556d4394e24 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:14.153838 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:14.210173 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c5976b-bc28-464e-8f9e-3556d4394e24 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:14.244824 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830634c2-0131-42fa-9521-b1b96c70b888 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:14.245748 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830634c2-0131-42fa-9521-b1b96c70b888 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:14.246035 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830634c2-0131-42fa-9521-b1b96c70b888 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:14.246310 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830634c2-0131-42fa-9521-b1b96c70b888 None admin] PATH_INFO: `/v3/projects/a596189576264bfc9638ca33f184759c` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:14.247552 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830634c2-0131-42fa-9521-b1b96c70b888 None admin] RBAC: Authorizing `identity:get_project(project_id=a596189576264bfc9638ca33f184759c)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:14.249010 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830634c2-0131-42fa-9521-b1b96c70b888 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:14.251028 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 110/233] 10.222.0.26 () {56 vars in 1186 bytes} [Thu Jul 14 09:04:14 2022] GET /identity/v3/projects/a596189576264bfc9638ca33f184759c => generated 302 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:04:14.256557 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd277e1b-2ccd-40fe-8e84-8b3a15b4ae5e None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:14.385318 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd277e1b-2ccd-40fe-8e84-8b3a15b4ae5e None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:14.420499 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d92a87f-57bf-4aa2-b471-a0651d23093b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:14.421405 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d92a87f-57bf-4aa2-b471-a0651d23093b None admin] REQUEST_METHOD: `PUT` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:14.421689 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d92a87f-57bf-4aa2-b471-a0651d23093b None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:14.421968 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d92a87f-57bf-4aa2-b471-a0651d23093b None admin] PATH_INFO: `/v3/projects/a596189576264bfc9638ca33f184759c/users/00e19d5626ee486082115e1a5c2d2876/roles/c4bab4a02e5c4f5f97a024e844a0c072` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:14.424230 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d92a87f-57bf-4aa2-b471-a0651d23093b None admin] RBAC: Authorizing `identity:create_grant(project_id=a596189576264bfc9638ca33f184759c, user_id=00e19d5626ee486082115e1a5c2d2876, role_id=c4bab4a02e5c4f5f97a024e844a0c072)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:14.425694 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d92a87f-57bf-4aa2-b471-a0651d23093b None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:14.450630 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 124/234] 10.222.0.26 () {58 vars in 1439 bytes} [Thu Jul 14 09:04:14 2022] PUT /identity/v3/projects/a596189576264bfc9638ca33f184759c/users/00e19d5626ee486082115e1a5c2d2876/roles/c4bab4a02e5c4f5f97a024e844a0c072 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:04:15.345364 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac60f72-a8fd-4449-bd05-391cb9823033 None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:15.345919 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac60f72-a8fd-4449-bd05-391cb9823033 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:15.345919 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac60f72-a8fd-4449-bd05-391cb9823033 None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:15.347556 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 111/235] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:04:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:04:15.352513 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603778d0-5aa2-439e-9b3e-313849395d4b None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:15.352910 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603778d0-5aa2-439e-9b3e-313849395d4b None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:15.353087 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603778d0-5aa2-439e-9b3e-313849395d4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:15.364323 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:15.380142 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-603778d0-5aa2-439e-9b3e-313849395d4b None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:04:15.381330 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-603778d0-5aa2-439e-9b3e-313849395d4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:04:15.430568 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6718eb-6297-4796-b378-6f9d466c4e9b None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:15.431209 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6718eb-6297-4796-b378-6f9d466c4e9b None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:15.431711 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6718eb-6297-4796-b378-6f9d466c4e9b None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:15.433305 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 112/236] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:04:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 09:04:15.438286 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a14ee30-befb-41c9-9857-e4a8f605b644 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:15.438797 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a14ee30-befb-41c9-9857-e4a8f605b644 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:15.439307 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a14ee30-befb-41c9-9857-e4a8f605b644 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:15.456897 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-0a14ee30-befb-41c9-9857-e4a8f605b644 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:04:15.458365 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-0a14ee30-befb-41c9-9857-e4a8f605b644 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:04:15.463652 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 125/237] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:04:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 09:04:15.485782 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603778d0-5aa2-439e-9b3e-313849395d4b None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:15.565737 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 113/238] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:04:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 09:04:15.594457 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a14ee30-befb-41c9-9857-e4a8f605b644 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:15.632673 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603778d0-5aa2-439e-9b3e-313849395d4b None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:15.662982 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:15.700086 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5b3c9a-d58a-4de6-852d-94ea40265051 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:15.700897 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5b3c9a-d58a-4de6-852d-94ea40265051 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:15.701174 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5b3c9a-d58a-4de6-852d-94ea40265051 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:15.701447 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5b3c9a-d58a-4de6-852d-94ea40265051 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:15.711212 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:15.720402 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5b3c9a-d58a-4de6-852d-94ea40265051 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 09:04:15.734565 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-ab5b3c9a-d58a-4de6-852d-94ea40265051 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 09:04:15.734565 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:04:15.734565 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:04:15.734565 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:04:15.734565 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:04:15.734565 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:04:15.734565 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:04:15.734565 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:04:15.734565 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:04:15.734565 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:04:15.734565 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:04:15.734565 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:04:15.734565 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 09:04:15.734565 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 09:04:15.734565 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 09:04:15.734565 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 09:04:15.735516 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 09:04:15.735516 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 09:04:15.735516 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 09:04:15.735516 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 09:04:15.735516 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:04:15.735516 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:04:15.735516 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:04:15.735516 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:04:15.735516 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:04:15.735516 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:04:15.735516 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:04:15.735516 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 14 09:04:15.735516 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:04:15.735516 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:04:15.735516 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:04:15.735516 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:04:15.736719 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:04:15.736719 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:04:15.736719 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:04:15.736719 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:04:15.736719 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 09:04:15.736719 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 09:04:15.736719 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 09:04:15.736719 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 09:04:15.736719 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 09:04:15.736719 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 09:04:15.736719 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 09:04:15.736719 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 14 09:04:15.736719 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 126/239] 10.222.0.26 () {56 vars in 1099 bytes} [Thu Jul 14 09:04:15 2022] GET /identity/v3/roles/reader => generated 84 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:04:15.738713 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a14ee30-befb-41c9-9857-e4a8f605b644 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:15.748145 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5b3c9a-d58a-4de6-852d-94ea40265051 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:15.808737 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59b172f-3e90-4f06-be1a-1f5b75c6b71d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:15.809540 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59b172f-3e90-4f06-be1a-1f5b75c6b71d None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:15.809818 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59b172f-3e90-4f06-be1a-1f5b75c6b71d None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:15.810089 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59b172f-3e90-4f06-be1a-1f5b75c6b71d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:15.811956 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59b172f-3e90-4f06-be1a-1f5b75c6b71d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:15.813403 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59b172f-3e90-4f06-be1a-1f5b75c6b71d None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:15.816059 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 114/240] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jul 14 09:04:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:04:15.824051 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59b172f-3e90-4f06-be1a-1f5b75c6b71d None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:15.926149 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59b172f-3e90-4f06-be1a-1f5b75c6b71d None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:15.944590 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:15.961510 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d096c10-b21a-4708-8a91-b912d3b44098 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:15.962636 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d096c10-b21a-4708-8a91-b912d3b44098 None admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:15.963018 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d096c10-b21a-4708-8a91-b912d3b44098 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:15.963429 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d096c10-b21a-4708-8a91-b912d3b44098 None admin] PATH_INFO: `/v3/users` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:15.964274 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d096c10-b21a-4708-8a91-b912d3b44098 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:15.965831 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d096c10-b21a-4708-8a91-b912d3b44098 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:15.989033 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 115/241] 10.222.0.26 () {60 vars in 1132 bytes} [Thu Jul 14 09:04:15 2022] POST /identity/v3/users => generated 293 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 09:04:15.992031 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5b3c9a-d58a-4de6-852d-94ea40265051 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:16.033547 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4645a48d-7f7d-431b-acf7-b6301323ce22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:16.034244 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4645a48d-7f7d-431b-acf7-b6301323ce22 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:16.034524 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4645a48d-7f7d-431b-acf7-b6301323ce22 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:16.034797 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4645a48d-7f7d-431b-acf7-b6301323ce22 None admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:16.035635 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4645a48d-7f7d-431b-acf7-b6301323ce22 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:04:16.035943 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4645a48d-7f7d-431b-acf7-b6301323ce22 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:16.037345 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4645a48d-7f7d-431b-acf7-b6301323ce22 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:16.047544 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 127/242] 10.222.0.26 () {56 vars in 1101 bytes} [Thu Jul 14 09:04:15 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:04:16.053210 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d096c10-b21a-4708-8a91-b912d3b44098 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:16.208439 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0d8cc5-ce74-4ff4-a6ef-c2bf41cd034b None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:16.209044 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0d8cc5-ce74-4ff4-a6ef-c2bf41cd034b None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:16.209582 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0d8cc5-ce74-4ff4-a6ef-c2bf41cd034b None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:16.211612 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 128/243] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:04:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 09:04:16.213138 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d096c10-b21a-4708-8a91-b912d3b44098 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:16.220899 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3f00e3-b5cd-46c2-8b32-069d6688e04b None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:16.221495 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3f00e3-b5cd-46c2-8b32-069d6688e04b None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:16.221987 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3f00e3-b5cd-46c2-8b32-069d6688e04b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:16.245295 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-eb3f00e3-b5cd-46c2-8b32-069d6688e04b None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:04:16.246986 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-eb3f00e3-b5cd-46c2-8b32-069d6688e04b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:04:16.271264 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687bc34d-6a2d-44c6-aa4f-2cfe1aac676e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:16.272152 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687bc34d-6a2d-44c6-aa4f-2cfe1aac676e None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:16.272430 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687bc34d-6a2d-44c6-aa4f-2cfe1aac676e None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:16.272738 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687bc34d-6a2d-44c6-aa4f-2cfe1aac676e None admin] PATH_INFO: `/v3/users/d7e17cf35330491aa753de3b5df8db3d` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:16.290294 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687bc34d-6a2d-44c6-aa4f-2cfe1aac676e None admin] RBAC: Authorizing `identity:get_user(user_id=d7e17cf35330491aa753de3b5df8db3d)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:16.291932 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687bc34d-6a2d-44c6-aa4f-2cfe1aac676e None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:16.294134 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 116/244] 10.222.0.26 () {56 vars in 1177 bytes} [Thu Jul 14 09:04:16 2022] GET /identity/v3/users/d7e17cf35330491aa753de3b5df8db3d => generated 297 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:04:16.299842 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687bc34d-6a2d-44c6-aa4f-2cfe1aac676e None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:16.321132 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:16.332924 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:16.347954 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 129/245] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:04:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 09:04:16.381152 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3f00e3-b5cd-46c2-8b32-069d6688e04b None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:16.433799 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687bc34d-6a2d-44c6-aa4f-2cfe1aac676e None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:16.479786 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec4afa2-8ba2-4d66-99f1-bff3e80ecc76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:16.480451 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec4afa2-8ba2-4d66-99f1-bff3e80ecc76 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:16.481854 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec4afa2-8ba2-4d66-99f1-bff3e80ecc76 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:16.481854 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec4afa2-8ba2-4d66-99f1-bff3e80ecc76 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:16.505914 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3f00e3-b5cd-46c2-8b32-069d6688e04b None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:16.547116 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0414c686-35f3-4990-a901-c08b04a8592f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:16.547953 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0414c686-35f3-4990-a901-c08b04a8592f None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:16.548234 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0414c686-35f3-4990-a901-c08b04a8592f None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:16.548498 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0414c686-35f3-4990-a901-c08b04a8592f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:16.561126 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0414c686-35f3-4990-a901-c08b04a8592f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 09:04:16.566776 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec4afa2-8ba2-4d66-99f1-bff3e80ecc76 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:16.570186 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec4afa2-8ba2-4d66-99f1-bff3e80ecc76 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:16.570330 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-0414c686-35f3-4990-a901-c08b04a8592f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 09:04:16.570330 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:04:16.570330 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:04:16.570330 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:04:16.570330 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:04:16.570330 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:04:16.570330 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:04:16.570330 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:04:16.570330 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:04:16.570330 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:04:16.570330 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:04:16.570330 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:04:16.570330 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 09:04:16.570330 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 09:04:16.570330 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 09:04:16.570330 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 09:04:16.571234 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 09:04:16.571234 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 09:04:16.571234 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 09:04:16.571234 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 09:04:16.571234 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:04:16.571234 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:04:16.571234 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:04:16.571234 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:04:16.571234 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:04:16.571234 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:04:16.571234 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:04:16.571234 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 14 09:04:16.571234 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:04:16.571234 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:04:16.571234 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:04:16.571234 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:04:16.572067 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:04:16.572067 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:04:16.572067 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:04:16.572067 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:04:16.572067 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 09:04:16.572067 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 09:04:16.572067 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 09:04:16.572067 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 09:04:16.572067 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 09:04:16.572067 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 09:04:16.572067 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 09:04:16.572067 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 14 09:04:16.572067 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 130/246] 10.222.0.26 () {56 vars in 1096 bytes} [Thu Jul 14 09:04:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 09:04:16.572740 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c548fa-d2d9-4ccf-b8ec-fe7bcac9dc41 None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:16.573509 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c548fa-d2d9-4ccf-b8ec-fe7bcac9dc41 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:16.573509 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c548fa-d2d9-4ccf-b8ec-fe7bcac9dc41 None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:16.574393 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 131/247] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:04:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:04:16.579926 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c548fa-d2d9-4ccf-b8ec-fe7bcac9dc41 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:16.602937 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:16.635710 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:16.694019 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 117/248] 10.222.0.26 () {58 vars in 1282 bytes} [Thu Jul 14 09:04:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 14 09:04:16.695896 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87accdcb-9524-45a6-9117-dd4bf5942706 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:16.696473 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87accdcb-9524-45a6-9117-dd4bf5942706 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:16.696915 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87accdcb-9524-45a6-9117-dd4bf5942706 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:16.707918 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c548fa-d2d9-4ccf-b8ec-fe7bcac9dc41 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:16.715777 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-87accdcb-9524-45a6-9117-dd4bf5942706 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:04:16.717274 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-87accdcb-9524-45a6-9117-dd4bf5942706 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:04:16.751725 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d66107-f5ba-41d1-a257-8566ec4df92a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:16.752921 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d66107-f5ba-41d1-a257-8566ec4df92a None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:16.753752 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d66107-f5ba-41d1-a257-8566ec4df92a None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:16.754467 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d66107-f5ba-41d1-a257-8566ec4df92a None admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:16.755433 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d66107-f5ba-41d1-a257-8566ec4df92a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:04:16.755908 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d66107-f5ba-41d1-a257-8566ec4df92a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:16.757475 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d66107-f5ba-41d1-a257-8566ec4df92a None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:16.769017 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 132/249] 10.222.0.26 () {56 vars in 1099 bytes} [Thu Jul 14 09:04:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:04:16.770431 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d66107-f5ba-41d1-a257-8566ec4df92a None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:16.800375 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 118/250] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:04:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 09:04:16.801844 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87accdcb-9524-45a6-9117-dd4bf5942706 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:16.869343 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:16.901961 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d66107-f5ba-41d1-a257-8566ec4df92a None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:16.913802 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:16.936645 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87accdcb-9524-45a6-9117-dd4bf5942706 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:16.942258 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350e0e10-783c-4a2a-811c-348c4b4bf5ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:16.943270 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350e0e10-783c-4a2a-811c-348c4b4bf5ce None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:16.943655 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350e0e10-783c-4a2a-811c-348c4b4bf5ce None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:16.944027 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350e0e10-783c-4a2a-811c-348c4b4bf5ce None admin] PATH_INFO: `/v3/projects/a596189576264bfc9638ca33f184759c` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:16.945375 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350e0e10-783c-4a2a-811c-348c4b4bf5ce None admin] RBAC: Authorizing `identity:get_project(project_id=a596189576264bfc9638ca33f184759c)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:16.947058 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350e0e10-783c-4a2a-811c-348c4b4bf5ce None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:16.949034 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 133/251] 10.222.0.26 () {56 vars in 1186 bytes} [Thu Jul 14 09:04:16 2022] GET /identity/v3/projects/a596189576264bfc9638ca33f184759c => generated 302 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:04:16.949974 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350e0e10-783c-4a2a-811c-348c4b4bf5ce None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:16.976500 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4d4084-4e51-42be-94c7-0c0c7c1afc9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:16.977877 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4d4084-4e51-42be-94c7-0c0c7c1afc9f None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:16.978322 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4d4084-4e51-42be-94c7-0c0c7c1afc9f None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:16.978877 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4d4084-4e51-42be-94c7-0c0c7c1afc9f None admin] PATH_INFO: `/v3/users/00e19d5626ee486082115e1a5c2d2876` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:16.980525 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4d4084-4e51-42be-94c7-0c0c7c1afc9f None admin] RBAC: Authorizing `identity:get_user(user_id=00e19d5626ee486082115e1a5c2d2876)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:16.982128 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4d4084-4e51-42be-94c7-0c0c7c1afc9f None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:16.984078 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 119/252] 10.222.0.26 () {56 vars in 1177 bytes} [Thu Jul 14 09:04:16 2022] GET /identity/v3/users/00e19d5626ee486082115e1a5c2d2876 => generated 283 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:04:16.985125 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4d4084-4e51-42be-94c7-0c0c7c1afc9f None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:17.078251 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350e0e10-783c-4a2a-811c-348c4b4bf5ce None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:17.126191 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26716395-0b8f-43be-af33-e39df1618801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:17.127004 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26716395-0b8f-43be-af33-e39df1618801 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:17.127317 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26716395-0b8f-43be-af33-e39df1618801 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:17.127594 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26716395-0b8f-43be-af33-e39df1618801 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:17.139617 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26716395-0b8f-43be-af33-e39df1618801 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 09:04:17.148466 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-26716395-0b8f-43be-af33-e39df1618801 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 09:04:17.148466 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:04:17.148466 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:04:17.148466 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:04:17.148466 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:04:17.148466 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:04:17.148466 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:04:17.148466 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:04:17.148466 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:04:17.148466 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:04:17.148466 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:04:17.148466 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:04:17.148466 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 09:04:17.148466 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 09:04:17.148466 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 09:04:17.148466 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 09:04:17.149376 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 09:04:17.149376 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 09:04:17.149376 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 09:04:17.149376 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 09:04:17.149376 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:04:17.149376 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:04:17.149376 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:04:17.149376 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:04:17.149376 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:04:17.149376 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:04:17.149376 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:04:17.149376 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 14 09:04:17.149376 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:04:17.149376 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:04:17.149376 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:04:17.149376 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:04:17.150599 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:04:17.150599 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:04:17.150599 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:04:17.150599 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:04:17.150599 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 09:04:17.150599 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 09:04:17.150599 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 09:04:17.150599 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 09:04:17.150599 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 09:04:17.150599 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 09:04:17.150599 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 09:04:17.150599 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 14 09:04:17.150599 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 134/253] 10.222.0.26 () {56 vars in 1099 bytes} [Thu Jul 14 09:04:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:04:17.151301 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26716395-0b8f-43be-af33-e39df1618801 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:17.197856 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:17.201883 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4d4084-4e51-42be-94c7-0c0c7c1afc9f None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:17.230926 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:17.245460 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cdf305-ab53-4b90-8e7b-ec69f81337d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:17.246427 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cdf305-ab53-4b90-8e7b-ec69f81337d6 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:17.246714 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cdf305-ab53-4b90-8e7b-ec69f81337d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:17.247287 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cdf305-ab53-4b90-8e7b-ec69f81337d6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:17.248141 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cdf305-ab53-4b90-8e7b-ec69f81337d6 None admin] RBAC: Adding query filter params (role.id=9c412baac5914ce6a40ad7d3f0f0a0d6, scope.project.id=a596189576264bfc9638ca33f184759c, user.id=d7e17cf35330491aa753de3b5df8db3d) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:04:17.248454 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cdf305-ab53-4b90-8e7b-ec69f81337d6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:17.250552 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cdf305-ab53-4b90-8e7b-ec69f81337d6 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:17.257985 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 120/254] 10.222.0.26 () {56 vars in 1374 bytes} [Thu Jul 14 09:04:16 2022] GET /identity/v3/role_assignments?user.id=d7e17cf35330491aa753de3b5df8db3d&scope.project.id=a596189576264bfc9638ca33f184759c&role.id=9c412baac5914ce6a40ad7d3f0f0a0d6 => generated 259 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:04:17.258886 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cdf305-ab53-4b90-8e7b-ec69f81337d6 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:17.281426 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26716395-0b8f-43be-af33-e39df1618801 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:17.356000 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb13ca9-5ff4-4122-bf63-2176cc935687 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:17.356589 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb13ca9-5ff4-4122-bf63-2176cc935687 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:17.356873 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb13ca9-5ff4-4122-bf63-2176cc935687 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:17.357153 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb13ca9-5ff4-4122-bf63-2176cc935687 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:17.383380 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cdf305-ab53-4b90-8e7b-ec69f81337d6 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:17.424194 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e31cf4b-e06d-4aca-87ec-4d7164da36b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:17.425374 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e31cf4b-e06d-4aca-87ec-4d7164da36b5 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:17.425810 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e31cf4b-e06d-4aca-87ec-4d7164da36b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:17.426541 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e31cf4b-e06d-4aca-87ec-4d7164da36b5 None admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:17.427482 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e31cf4b-e06d-4aca-87ec-4d7164da36b5 None admin] RBAC: Adding query filter params (name=member) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:04:17.427932 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e31cf4b-e06d-4aca-87ec-4d7164da36b5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:17.429601 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e31cf4b-e06d-4aca-87ec-4d7164da36b5 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:17.440117 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 121/255] 10.222.0.26 () {56 vars in 1101 bytes} [Thu Jul 14 09:04:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:04:17.442371 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4c1ff7-b6e2-4ed7-9603-a9536db34d3d None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:17.442799 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4c1ff7-b6e2-4ed7-9603-a9536db34d3d None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:17.443238 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4c1ff7-b6e2-4ed7-9603-a9536db34d3d None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:17.445156 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 122/256] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:04:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:04:17.447415 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4c1ff7-b6e2-4ed7-9603-a9536db34d3d None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:17.509979 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:17.525179 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb13ca9-5ff4-4122-bf63-2176cc935687 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:17.527229 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb13ca9-5ff4-4122-bf63-2176cc935687 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:17.569766 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4c1ff7-b6e2-4ed7-9603-a9536db34d3d None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:17.607776 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbcfb62-256c-4490-8a50-fa2d8d6d68f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:17.608969 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbcfb62-256c-4490-8a50-fa2d8d6d68f4 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:17.610131 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbcfb62-256c-4490-8a50-fa2d8d6d68f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:17.610549 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbcfb62-256c-4490-8a50-fa2d8d6d68f4 None admin] PATH_INFO: `/v3/users/06a6d5cbc4324313a9f8745928cd5ca8` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:17.610826 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:17.613687 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbcfb62-256c-4490-8a50-fa2d8d6d68f4 None admin] RBAC: Authorizing `identity:get_user(user_id=06a6d5cbc4324313a9f8745928cd5ca8)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:17.615151 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbcfb62-256c-4490-8a50-fa2d8d6d68f4 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:17.622456 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 123/257] 10.222.0.26 () {56 vars in 1177 bytes} [Thu Jul 14 09:04:17 2022] GET /identity/v3/users/06a6d5cbc4324313a9f8745928cd5ca8 => generated 297 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:04:17.624296 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b60f73-a29c-4cbf-92bf-8e9c32d6b36c None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:17.624781 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b60f73-a29c-4cbf-92bf-8e9c32d6b36c None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:17.627488 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b60f73-a29c-4cbf-92bf-8e9c32d6b36c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:17.654244 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-d5b60f73-a29c-4cbf-92bf-8e9c32d6b36c None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:04:17.656124 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-d5b60f73-a29c-4cbf-92bf-8e9c32d6b36c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:04:17.759299 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 135/258] 10.222.0.26 () {58 vars in 1282 bytes} [Thu Jul 14 09:04:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 608 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 09:04:17.761226 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb13ca9-5ff4-4122-bf63-2176cc935687 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:17.792983 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 124/259] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:04:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 09:04:17.793872 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b60f73-a29c-4cbf-92bf-8e9c32d6b36c None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:17.880449 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb13ca9-5ff4-4122-bf63-2176cc935687 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:17.890427 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:17.915430 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f58df44-8c37-4869-9c20-d42a315369e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:17.916328 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f58df44-8c37-4869-9c20-d42a315369e7 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:17.916612 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f58df44-8c37-4869-9c20-d42a315369e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:17.916890 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f58df44-8c37-4869-9c20-d42a315369e7 None admin] PATH_INFO: `/v3/projects/a596189576264bfc9638ca33f184759c` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:17.918324 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f58df44-8c37-4869-9c20-d42a315369e7 None admin] RBAC: Authorizing `identity:get_project(project_id=a596189576264bfc9638ca33f184759c)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:17.919974 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f58df44-8c37-4869-9c20-d42a315369e7 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:17.922177 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 136/260] 10.222.0.26 () {56 vars in 1186 bytes} [Thu Jul 14 09:04:17 2022] GET /identity/v3/projects/a596189576264bfc9638ca33f184759c => generated 302 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:04:17.923106 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f58df44-8c37-4869-9c20-d42a315369e7 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:17.927426 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:18.009889 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b60f73-a29c-4cbf-92bf-8e9c32d6b36c None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:18.029629 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f58df44-8c37-4869-9c20-d42a315369e7 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:18.069112 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2e2536-68c9-4f02-85de-ce6ed9f36d4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:18.069933 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2e2536-68c9-4f02-85de-ce6ed9f36d4f None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:18.070217 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2e2536-68c9-4f02-85de-ce6ed9f36d4f None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:18.070493 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2e2536-68c9-4f02-85de-ce6ed9f36d4f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:18.081278 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2e2536-68c9-4f02-85de-ce6ed9f36d4f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 09:04:18.085556 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f3ac98-1b87-441d-a5f1-52b0e7dc3304 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:18.086439 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f3ac98-1b87-441d-a5f1-52b0e7dc3304 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:18.086723 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f3ac98-1b87-441d-a5f1-52b0e7dc3304 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:18.086997 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f3ac98-1b87-441d-a5f1-52b0e7dc3304 None admin] PATH_INFO: `/v3/projects/a596189576264bfc9638ca33f184759c` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:18.089083 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f3ac98-1b87-441d-a5f1-52b0e7dc3304 None admin] RBAC: Authorizing `identity:get_project(project_id=a596189576264bfc9638ca33f184759c)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:18.089392 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-cd2e2536-68c9-4f02-85de-ce6ed9f36d4f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 09:04:18.089392 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:04:18.089392 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:04:18.089392 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:04:18.089392 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:04:18.089392 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:04:18.089392 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:04:18.089392 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:04:18.089392 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:04:18.089392 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:04:18.089392 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:04:18.089392 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:04:18.089392 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 09:04:18.089392 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 09:04:18.089392 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 09:04:18.089392 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 09:04:18.090268 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 09:04:18.090268 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 09:04:18.090268 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 09:04:18.090268 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 09:04:18.090268 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:04:18.090268 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:04:18.090268 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:04:18.090268 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:04:18.090268 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:04:18.090268 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:04:18.090268 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:04:18.090268 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 14 09:04:18.090268 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:04:18.090268 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:04:18.090268 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:04:18.090268 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:04:18.091083 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:04:18.091083 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:04:18.091083 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:04:18.091083 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:04:18.091083 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 09:04:18.091083 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 09:04:18.091083 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 09:04:18.091083 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 09:04:18.091083 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 09:04:18.091083 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 09:04:18.091083 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 09:04:18.091083 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 14 09:04:18.091732 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f3ac98-1b87-441d-a5f1-52b0e7dc3304 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:18.092320 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 137/261] 10.222.0.26 () {56 vars in 1114 bytes} [Thu Jul 14 09:04:17 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:04:18.092320 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2e2536-68c9-4f02-85de-ce6ed9f36d4f None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:18.092610 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 125/262] 10.222.0.26 () {56 vars in 1186 bytes} [Thu Jul 14 09:04:17 2022] GET /identity/v3/projects/a596189576264bfc9638ca33f184759c => generated 302 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:04:18.097548 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f3ac98-1b87-441d-a5f1-52b0e7dc3304 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:18.169879 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:18.216907 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2e2536-68c9-4f02-85de-ce6ed9f36d4f None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:18.262198 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e12b0c3-4e66-4ad6-b0eb-caf4566e64ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:18.263106 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e12b0c3-4e66-4ad6-b0eb-caf4566e64ac None admin] REQUEST_METHOD: `PUT` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:18.263410 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e12b0c3-4e66-4ad6-b0eb-caf4566e64ac None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:18.263687 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e12b0c3-4e66-4ad6-b0eb-caf4566e64ac None admin] PATH_INFO: `/v3/projects/a596189576264bfc9638ca33f184759c/users/06a6d5cbc4324313a9f8745928cd5ca8/roles/d1691322ce6a4edab7ce722edf0f2988` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:18.266727 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e12b0c3-4e66-4ad6-b0eb-caf4566e64ac None admin] RBAC: Authorizing `identity:create_grant(project_id=a596189576264bfc9638ca33f184759c, user_id=06a6d5cbc4324313a9f8745928cd5ca8, role_id=d1691322ce6a4edab7ce722edf0f2988)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:18.268224 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e12b0c3-4e66-4ad6-b0eb-caf4566e64ac None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:18.300777 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f3ac98-1b87-441d-a5f1-52b0e7dc3304 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:18.309250 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 138/263] 10.222.0.26 () {58 vars in 1439 bytes} [Thu Jul 14 09:04:18 2022] PUT /identity/v3/projects/a596189576264bfc9638ca33f184759c/users/06a6d5cbc4324313a9f8745928cd5ca8/roles/d1691322ce6a4edab7ce722edf0f2988 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:04:18.310170 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e12b0c3-4e66-4ad6-b0eb-caf4566e64ac None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:18.335620 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:18.379218 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0410290-cafb-4e48-8795-25d61cca2908 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:18.379218 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0410290-cafb-4e48-8795-25d61cca2908 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:18.379218 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0410290-cafb-4e48-8795-25d61cca2908 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:18.379218 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0410290-cafb-4e48-8795-25d61cca2908 None admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:18.379634 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0410290-cafb-4e48-8795-25d61cca2908 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:04:18.379634 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0410290-cafb-4e48-8795-25d61cca2908 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:18.379634 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0410290-cafb-4e48-8795-25d61cca2908 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:18.395228 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 126/264] 10.222.0.26 () {56 vars in 1111 bytes} [Thu Jul 14 09:04:18 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 09:04:18.396746 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0410290-cafb-4e48-8795-25d61cca2908 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:18.553241 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e12b0c3-4e66-4ad6-b0eb-caf4566e64ac None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:18.572358 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0410290-cafb-4e48-8795-25d61cca2908 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:18.629682 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:18.636420 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d362d133-15dc-4871-9b59-24fe186ed401 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:18.637328 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d362d133-15dc-4871-9b59-24fe186ed401 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:18.637615 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d362d133-15dc-4871-9b59-24fe186ed401 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:18.637897 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d362d133-15dc-4871-9b59-24fe186ed401 None admin] PATH_INFO: `/v3/users/00e19d5626ee486082115e1a5c2d2876` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:18.640531 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d362d133-15dc-4871-9b59-24fe186ed401 None admin] RBAC: Authorizing `identity:get_user(user_id=00e19d5626ee486082115e1a5c2d2876)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:18.644807 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d362d133-15dc-4871-9b59-24fe186ed401 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:18.647814 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 127/265] 10.222.0.26 () {56 vars in 1177 bytes} [Thu Jul 14 09:04:18 2022] GET /identity/v3/users/00e19d5626ee486082115e1a5c2d2876 => generated 283 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 09:04:18.654478 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d362d133-15dc-4871-9b59-24fe186ed401 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:18.659215 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cd123e-e1ba-45b9-adcc-fa9d4efbb6bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:18.659894 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cd123e-e1ba-45b9-adcc-fa9d4efbb6bc None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:18.660172 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cd123e-e1ba-45b9-adcc-fa9d4efbb6bc None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:18.660446 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cd123e-e1ba-45b9-adcc-fa9d4efbb6bc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:18.661287 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63cd123e-e1ba-45b9-adcc-fa9d4efbb6bc None admin] RBAC: Adding query filter params (role.id=7b549d58d1724395b343609d895c80d0, scope.project.id=a596189576264bfc9638ca33f184759c, user.id=00e19d5626ee486082115e1a5c2d2876) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:04:18.661595 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63cd123e-e1ba-45b9-adcc-fa9d4efbb6bc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:18.671422 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63cd123e-e1ba-45b9-adcc-fa9d4efbb6bc None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:18.689670 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 139/266] 10.222.0.26 () {56 vars in 1374 bytes} [Thu Jul 14 09:04:18 2022] GET /identity/v3/role_assignments?user.id=00e19d5626ee486082115e1a5c2d2876&scope.project.id=a596189576264bfc9638ca33f184759c&role.id=7b549d58d1724395b343609d895c80d0 => generated 608 bytes in 380 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:04:18.739565 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:18.817241 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d362d133-15dc-4871-9b59-24fe186ed401 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:18.860770 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509d2895-4789-4523-9446-d72807d480b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:18.860770 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509d2895-4789-4523-9446-d72807d480b7 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:18.861067 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509d2895-4789-4523-9446-d72807d480b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:18.861294 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509d2895-4789-4523-9446-d72807d480b7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:18.932499 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509d2895-4789-4523-9446-d72807d480b7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:18.933984 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509d2895-4789-4523-9446-d72807d480b7 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:18.979796 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:19.000194 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d679ae6-d47f-4d3f-a45f-ccdedd8142eb None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:19.000798 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d679ae6-d47f-4d3f-a45f-ccdedd8142eb None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:19.001303 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d679ae6-d47f-4d3f-a45f-ccdedd8142eb None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:19.003213 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 140/267] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:04:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 09:04:19.008743 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a2e163-0a76-440d-9953-f9b5076f8ea0 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:19.009302 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a2e163-0a76-440d-9953-f9b5076f8ea0 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:19.009770 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a2e163-0a76-440d-9953-f9b5076f8ea0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:19.041562 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-42a2e163-0a76-440d-9953-f9b5076f8ea0 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:04:19.043077 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-42a2e163-0a76-440d-9953-f9b5076f8ea0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:04:19.045283 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 128/268] 10.222.0.26 () {58 vars in 1282 bytes} [Thu Jul 14 09:04:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 14 09:04:19.053289 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509d2895-4789-4523-9446-d72807d480b7 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:19.168411 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 141/269] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:04:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 09:04:19.170382 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704cbc22-1cb2-4df9-b3ef-46eaf2e5ee12 None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:19.171031 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704cbc22-1cb2-4df9-b3ef-46eaf2e5ee12 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:19.173282 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704cbc22-1cb2-4df9-b3ef-46eaf2e5ee12 None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:19.174868 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 142/270] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:04:19 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:04:19.180859 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb49615-854b-4f55-be74-160622ff9fde None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:19.181438 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb49615-854b-4f55-be74-160622ff9fde None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:19.182001 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb49615-854b-4f55-be74-160622ff9fde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:19.208393 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-3fb49615-854b-4f55-be74-160622ff9fde None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:04:19.210043 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-3fb49615-854b-4f55-be74-160622ff9fde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:04:19.223699 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509d2895-4789-4523-9446-d72807d480b7 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:19.266513 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83200b40-4f0e-4e0d-978f-cf843fc97735 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:19.267883 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83200b40-4f0e-4e0d-978f-cf843fc97735 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:19.268339 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83200b40-4f0e-4e0d-978f-cf843fc97735 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:19.268790 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83200b40-4f0e-4e0d-978f-cf843fc97735 None admin] PATH_INFO: `/v3/projects/a596189576264bfc9638ca33f184759c` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:19.270865 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83200b40-4f0e-4e0d-978f-cf843fc97735 None admin] RBAC: Authorizing `identity:get_project(project_id=a596189576264bfc9638ca33f184759c)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:19.272578 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83200b40-4f0e-4e0d-978f-cf843fc97735 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:19.279256 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 129/271] 10.222.0.26 () {56 vars in 1186 bytes} [Thu Jul 14 09:04:19 2022] GET /identity/v3/projects/a596189576264bfc9638ca33f184759c => generated 302 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:04:19.279256 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83200b40-4f0e-4e0d-978f-cf843fc97735 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:19.292721 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:19.316460 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 143/272] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:04:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 09:04:19.317338 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb49615-854b-4f55-be74-160622ff9fde None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:19.375067 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:19.412746 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83200b40-4f0e-4e0d-978f-cf843fc97735 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:19.442804 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb49615-854b-4f55-be74-160622ff9fde None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:19.453735 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e465b7-2aab-4b97-867c-2ee3d4ff3724 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:19.455150 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e465b7-2aab-4b97-867c-2ee3d4ff3724 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:19.455870 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e465b7-2aab-4b97-867c-2ee3d4ff3724 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:19.456292 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e465b7-2aab-4b97-867c-2ee3d4ff3724 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:19.471161 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e465b7-2aab-4b97-867c-2ee3d4ff3724 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 09:04:19.478560 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963c1baa-fa9a-4956-b3d4-d17b03752ec5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:19.479427 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963c1baa-fa9a-4956-b3d4-d17b03752ec5 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:19.479904 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963c1baa-fa9a-4956-b3d4-d17b03752ec5 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:19.480355 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963c1baa-fa9a-4956-b3d4-d17b03752ec5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:19.480764 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-68e465b7-2aab-4b97-867c-2ee3d4ff3724 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 09:04:19.480764 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:04:19.480764 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:04:19.480764 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:04:19.480764 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:04:19.480764 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:04:19.480764 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:04:19.480764 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:04:19.480764 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:04:19.480764 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:04:19.480764 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:04:19.480764 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:04:19.480764 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 09:04:19.480764 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 09:04:19.480764 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 09:04:19.480764 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 09:04:19.481822 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 09:04:19.481822 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 09:04:19.481822 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 09:04:19.481822 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 09:04:19.481822 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:04:19.481822 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:04:19.481822 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:04:19.481822 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:04:19.481822 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:04:19.481822 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:04:19.481822 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:04:19.481822 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 14 09:04:19.481822 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:04:19.481822 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:04:19.481822 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:04:19.481822 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:04:19.482699 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:04:19.482699 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:04:19.482699 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:04:19.482699 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:04:19.482699 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 09:04:19.482699 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 09:04:19.482699 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 09:04:19.482699 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 09:04:19.482699 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 09:04:19.482699 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 09:04:19.482699 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 09:04:19.482699 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 14 09:04:19.482699 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 130/273] 10.222.0.26 () {56 vars in 1099 bytes} [Thu Jul 14 09:04:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:04:19.483419 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e465b7-2aab-4b97-867c-2ee3d4ff3724 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:19.484403 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963c1baa-fa9a-4956-b3d4-d17b03752ec5 None admin] RBAC: Adding query filter params (role.id=c4bab4a02e5c4f5f97a024e844a0c072, scope.project.id=a596189576264bfc9638ca33f184759c, user.id=00e19d5626ee486082115e1a5c2d2876) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:04:19.484977 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963c1baa-fa9a-4956-b3d4-d17b03752ec5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:19.486690 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963c1baa-fa9a-4956-b3d4-d17b03752ec5 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:19.494022 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 144/274] 10.222.0.26 () {56 vars in 1374 bytes} [Thu Jul 14 09:04:19 2022] GET /identity/v3/role_assignments?user.id=00e19d5626ee486082115e1a5c2d2876&scope.project.id=a596189576264bfc9638ca33f184759c&role.id=c4bab4a02e5c4f5f97a024e844a0c072 => generated 608 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:04:19.495086 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963c1baa-fa9a-4956-b3d4-d17b03752ec5 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:19.634138 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:19.695718 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963c1baa-fa9a-4956-b3d4-d17b03752ec5 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:19.696233 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e465b7-2aab-4b97-867c-2ee3d4ff3724 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:19.713502 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:19.739919 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80b9cd6-c74c-4215-93a9-de70561b5b1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:19.741270 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80b9cd6-c74c-4215-93a9-de70561b5b1b None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:19.741551 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80b9cd6-c74c-4215-93a9-de70561b5b1b None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:19.741822 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80b9cd6-c74c-4215-93a9-de70561b5b1b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:19.743072 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80b9cd6-c74c-4215-93a9-de70561b5b1b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:19.743804 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95879430-0c53-40fa-be11-82276965a608 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:19.744473 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95879430-0c53-40fa-be11-82276965a608 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:19.744747 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95879430-0c53-40fa-be11-82276965a608 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:19.745013 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95879430-0c53-40fa-be11-82276965a608 None admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:19.745108 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80b9cd6-c74c-4215-93a9-de70561b5b1b None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:19.745766 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95879430-0c53-40fa-be11-82276965a608 None admin] RBAC: Adding query filter params (name=member) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:04:19.746067 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95879430-0c53-40fa-be11-82276965a608 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:19.746907 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 131/275] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jul 14 09:04:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:04:19.747532 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95879430-0c53-40fa-be11-82276965a608 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:19.747989 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611577fd-5551-4f23-b668-21be87ccc075 None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:19.748278 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611577fd-5551-4f23-b668-21be87ccc075 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:19.748557 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611577fd-5551-4f23-b668-21be87ccc075 None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:19.749804 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 132/276] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:04:19 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:04:19.756097 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611577fd-5551-4f23-b668-21be87ccc075 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:19.757076 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 145/277] 10.222.0.26 () {56 vars in 1101 bytes} [Thu Jul 14 09:04:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:04:19.758318 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa19d23-f33c-451f-9bf6-96d7a0cc1a6f None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:19.758606 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa19d23-f33c-451f-9bf6-96d7a0cc1a6f None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:19.758880 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa19d23-f33c-451f-9bf6-96d7a0cc1a6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:19.776732 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-7fa19d23-f33c-451f-9bf6-96d7a0cc1a6f None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:04:19.777905 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-7fa19d23-f33c-451f-9bf6-96d7a0cc1a6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:04:19.855686 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 146/278] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:04:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 09:04:19.856432 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa19d23-f33c-451f-9bf6-96d7a0cc1a6f None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:19.882427 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611577fd-5551-4f23-b668-21be87ccc075 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:19.918009 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4351a2b9-2262-4205-b9e6-8a4bd24fa467 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:19.918692 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4351a2b9-2262-4205-b9e6-8a4bd24fa467 None admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:19.918974 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4351a2b9-2262-4205-b9e6-8a4bd24fa467 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:19.919280 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4351a2b9-2262-4205-b9e6-8a4bd24fa467 None admin] PATH_INFO: `/v3/users` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:19.919995 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4351a2b9-2262-4205-b9e6-8a4bd24fa467 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:19.921520 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4351a2b9-2262-4205-b9e6-8a4bd24fa467 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:19.932232 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:19.945694 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 133/279] 10.222.0.26 () {60 vars in 1132 bytes} [Thu Jul 14 09:04:19 2022] POST /identity/v3/users => generated 293 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 09:04:19.946735 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4351a2b9-2262-4205-b9e6-8a4bd24fa467 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:19.978696 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa19d23-f33c-451f-9bf6-96d7a0cc1a6f None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:19.990349 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:20.035164 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934665fc-958d-4a7c-bb4d-d6973eb78d98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:20.036593 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934665fc-958d-4a7c-bb4d-d6973eb78d98 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:20.037052 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934665fc-958d-4a7c-bb4d-d6973eb78d98 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:20.037572 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934665fc-958d-4a7c-bb4d-d6973eb78d98 None admin] PATH_INFO: `/v3/users/617dad19c58948b38a5f12260c00d9af` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:20.061428 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934665fc-958d-4a7c-bb4d-d6973eb78d98 None admin] RBAC: Authorizing `identity:get_user(user_id=617dad19c58948b38a5f12260c00d9af)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:20.064301 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934665fc-958d-4a7c-bb4d-d6973eb78d98 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:20.067495 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 147/280] 10.222.0.26 () {56 vars in 1177 bytes} [Thu Jul 14 09:04:19 2022] GET /identity/v3/users/617dad19c58948b38a5f12260c00d9af => generated 293 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:04:20.069497 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4351a2b9-2262-4205-b9e6-8a4bd24fa467 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:20.074719 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934665fc-958d-4a7c-bb4d-d6973eb78d98 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:20.109099 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb393f81-3969-4687-9d65-8d4dc87633de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:20.110050 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb393f81-3969-4687-9d65-8d4dc87633de None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:20.110341 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb393f81-3969-4687-9d65-8d4dc87633de None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:20.110615 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb393f81-3969-4687-9d65-8d4dc87633de None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:20.123164 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb393f81-3969-4687-9d65-8d4dc87633de None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 09:04:20.132682 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-fb393f81-3969-4687-9d65-8d4dc87633de None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 09:04:20.132682 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:04:20.132682 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:04:20.132682 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:04:20.132682 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:04:20.132682 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:04:20.132682 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:04:20.132682 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:04:20.132682 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:04:20.132682 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:04:20.132682 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:04:20.132682 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:04:20.132682 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 09:04:20.132682 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 09:04:20.132682 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 09:04:20.132682 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 09:04:20.133588 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 09:04:20.133588 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 09:04:20.133588 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 09:04:20.133588 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 09:04:20.133588 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:04:20.133588 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:04:20.133588 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:04:20.133588 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:04:20.133588 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:04:20.133588 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:04:20.133588 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:04:20.133588 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 14 09:04:20.133588 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:04:20.133588 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:04:20.133588 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:04:20.133588 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:04:20.134393 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:04:20.134393 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:04:20.134393 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:04:20.134393 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:04:20.134393 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 09:04:20.134393 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 09:04:20.134393 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 09:04:20.134393 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 09:04:20.134393 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 09:04:20.134393 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 09:04:20.134393 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 09:04:20.134393 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 14 09:04:20.134393 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 134/281] 10.222.0.26 () {56 vars in 1099 bytes} [Thu Jul 14 09:04:19 2022] GET /identity/v3/roles/reader => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:04:20.139986 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb393f81-3969-4687-9d65-8d4dc87633de None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:20.201869 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934665fc-958d-4a7c-bb4d-d6973eb78d98 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:20.210175 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:20.248516 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24123163-f967-482a-84e4-01684ca0cb59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:20.249188 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24123163-f967-482a-84e4-01684ca0cb59 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:20.249470 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24123163-f967-482a-84e4-01684ca0cb59 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:20.249893 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24123163-f967-482a-84e4-01684ca0cb59 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:20.251075 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24123163-f967-482a-84e4-01684ca0cb59 None admin] RBAC: Adding query filter params (role.id=d1691322ce6a4edab7ce722edf0f2988, user.id=617dad19c58948b38a5f12260c00d9af, scope.system=all) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:04:20.251429 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24123163-f967-482a-84e4-01684ca0cb59 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:20.252895 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24123163-f967-482a-84e4-01684ca0cb59 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:20.259028 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 148/282] 10.222.0.26 () {56 vars in 1308 bytes} [Thu Jul 14 09:04:20 2022] GET /identity/v3/role_assignments?user.id=617dad19c58948b38a5f12260c00d9af&scope.system=all&role.id=d1691322ce6a4edab7ce722edf0f2988 => generated 226 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:04:20.260616 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb393f81-3969-4687-9d65-8d4dc87633de None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:20.299485 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36eea6f0-ea72-4a5a-816b-f326308038ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:20.300209 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36eea6f0-ea72-4a5a-816b-f326308038ff None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:20.300808 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36eea6f0-ea72-4a5a-816b-f326308038ff None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:20.301081 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36eea6f0-ea72-4a5a-816b-f326308038ff None admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:20.301845 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36eea6f0-ea72-4a5a-816b-f326308038ff None admin] RBAC: Adding query filter params (name=reader) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:04:20.302148 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36eea6f0-ea72-4a5a-816b-f326308038ff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:20.303637 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36eea6f0-ea72-4a5a-816b-f326308038ff None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:20.313743 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 135/283] 10.222.0.26 () {56 vars in 1101 bytes} [Thu Jul 14 09:04:20 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:04:20.319626 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24123163-f967-482a-84e4-01684ca0cb59 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:20.349809 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:20.434907 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24123163-f967-482a-84e4-01684ca0cb59 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:20.473564 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a869e0f-8685-46d3-85c4-153ceab935d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:20.474458 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a869e0f-8685-46d3-85c4-153ceab935d7 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:20.474773 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a869e0f-8685-46d3-85c4-153ceab935d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:20.475047 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a869e0f-8685-46d3-85c4-153ceab935d7 None admin] PATH_INFO: `/v3/users/d7e17cf35330491aa753de3b5df8db3d` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:20.477645 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a869e0f-8685-46d3-85c4-153ceab935d7 None admin] RBAC: Authorizing `identity:get_user(user_id=d7e17cf35330491aa753de3b5df8db3d)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:20.479113 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a869e0f-8685-46d3-85c4-153ceab935d7 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:20.480817 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 149/284] 10.222.0.26 () {56 vars in 1177 bytes} [Thu Jul 14 09:04:20 2022] GET /identity/v3/users/d7e17cf35330491aa753de3b5df8db3d => generated 297 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 09:04:20.486361 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36eea6f0-ea72-4a5a-816b-f326308038ff None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:20.601197 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36eea6f0-ea72-4a5a-816b-f326308038ff None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:20.623714 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:20.650183 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6406bac9-b37e-4e78-842d-0ffa11f56b3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:20.651831 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6406bac9-b37e-4e78-842d-0ffa11f56b3d None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:20.652291 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6406bac9-b37e-4e78-842d-0ffa11f56b3d None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:20.652739 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6406bac9-b37e-4e78-842d-0ffa11f56b3d None admin] PATH_INFO: `/v3/projects/a596189576264bfc9638ca33f184759c` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:20.659920 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6406bac9-b37e-4e78-842d-0ffa11f56b3d None admin] RBAC: Authorizing `identity:get_project(project_id=a596189576264bfc9638ca33f184759c)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:20.661604 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6406bac9-b37e-4e78-842d-0ffa11f56b3d None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:20.663914 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 136/285] 10.222.0.26 () {56 vars in 1186 bytes} [Thu Jul 14 09:04:20 2022] GET /identity/v3/projects/a596189576264bfc9638ca33f184759c => generated 302 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:04:20.672341 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a869e0f-8685-46d3-85c4-153ceab935d7 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:20.782526 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:20.793653 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a869e0f-8685-46d3-85c4-153ceab935d7 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:20.853098 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7174516c-970c-4576-890d-24f48dc25c12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:20.854002 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7174516c-970c-4576-890d-24f48dc25c12 None admin] REQUEST_METHOD: `PUT` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:20.854282 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7174516c-970c-4576-890d-24f48dc25c12 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:20.854556 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7174516c-970c-4576-890d-24f48dc25c12 None admin] PATH_INFO: `/v3/projects/a596189576264bfc9638ca33f184759c/users/d7e17cf35330491aa753de3b5df8db3d/roles/9c412baac5914ce6a40ad7d3f0f0a0d6` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:20.860504 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7174516c-970c-4576-890d-24f48dc25c12 None admin] RBAC: Authorizing `identity:create_grant(project_id=a596189576264bfc9638ca33f184759c, user_id=d7e17cf35330491aa753de3b5df8db3d, role_id=9c412baac5914ce6a40ad7d3f0f0a0d6)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:20.861954 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7174516c-970c-4576-890d-24f48dc25c12 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:20.880483 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 150/286] 10.222.0.26 () {58 vars in 1439 bytes} [Thu Jul 14 09:04:20 2022] PUT /identity/v3/projects/a596189576264bfc9638ca33f184759c/users/d7e17cf35330491aa753de3b5df8db3d/roles/9c412baac5914ce6a40ad7d3f0f0a0d6 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:04:21.006485 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f251d985-4c75-48bf-a5ec-21eb35c79113 None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:21.006861 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f251d985-4c75-48bf-a5ec-21eb35c79113 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:21.007225 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f251d985-4c75-48bf-a5ec-21eb35c79113 None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:21.009030 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 137/287] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:04:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:04:21.017974 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9558624a-fc9f-4e39-af80-c5637747e8d1 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:21.018257 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9558624a-fc9f-4e39-af80-c5637747e8d1 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:21.018534 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9558624a-fc9f-4e39-af80-c5637747e8d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:21.040383 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-9558624a-fc9f-4e39-af80-c5637747e8d1 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:04:21.041532 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-9558624a-fc9f-4e39-af80-c5637747e8d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:04:21.127671 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 151/288] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:04:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 09:04:21.153506 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f251d985-4c75-48bf-a5ec-21eb35c79113 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:21.297116 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f251d985-4c75-48bf-a5ec-21eb35c79113 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:21.348782 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f87c41-13ad-488d-a10a-ed02f4dfdc88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:21.349967 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f87c41-13ad-488d-a10a-ed02f4dfdc88 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:21.350435 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f87c41-13ad-488d-a10a-ed02f4dfdc88 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:21.351155 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f87c41-13ad-488d-a10a-ed02f4dfdc88 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:21.361987 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f87c41-13ad-488d-a10a-ed02f4dfdc88 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 09:04:21.370029 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-79f87c41-13ad-488d-a10a-ed02f4dfdc88 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 09:04:21.370029 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:04:21.370029 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:04:21.370029 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:04:21.370029 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:04:21.370029 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:04:21.370029 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:04:21.370029 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:04:21.370029 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:04:21.370029 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:04:21.370029 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:04:21.370029 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:04:21.370029 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 09:04:21.370029 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 09:04:21.370029 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 09:04:21.370029 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 09:04:21.370939 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 09:04:21.370939 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 09:04:21.370939 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 09:04:21.370939 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 09:04:21.370939 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:04:21.370939 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:04:21.370939 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:04:21.370939 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:04:21.370939 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:04:21.370939 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:04:21.370939 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:04:21.370939 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 14 09:04:21.370939 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:04:21.370939 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:04:21.370939 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:04:21.370939 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:04:21.371781 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:04:21.371781 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:04:21.371781 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:04:21.371781 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:04:21.371781 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 09:04:21.371781 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 09:04:21.371781 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 09:04:21.371781 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 09:04:21.371781 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 09:04:21.371781 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 09:04:21.371781 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 09:04:21.371781 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 14 09:04:21.374138 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 138/289] 10.222.0.26 () {56 vars in 1099 bytes} [Thu Jul 14 09:04:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:04:21.380041 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9558624a-fc9f-4e39-af80-c5637747e8d1 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:21.541230 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:21.566575 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9558624a-fc9f-4e39-af80-c5637747e8d1 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:21.616014 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b21ecfc-5def-4b55-abff-de410b6cc7e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:21.617049 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b21ecfc-5def-4b55-abff-de410b6cc7e5 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:21.617531 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b21ecfc-5def-4b55-abff-de410b6cc7e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:21.618000 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b21ecfc-5def-4b55-abff-de410b6cc7e5 None admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:21.618959 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b21ecfc-5def-4b55-abff-de410b6cc7e5 None admin] RBAC: Adding query filter params (name=member) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:04:21.619513 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b21ecfc-5def-4b55-abff-de410b6cc7e5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:21.621102 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b21ecfc-5def-4b55-abff-de410b6cc7e5 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:21.635387 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 152/290] 10.222.0.26 () {56 vars in 1101 bytes} [Thu Jul 14 09:04:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:04:21.641438 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f87c41-13ad-488d-a10a-ed02f4dfdc88 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:21.691560 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:21.774054 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f87c41-13ad-488d-a10a-ed02f4dfdc88 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:21.825837 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b750759-ae93-43ab-81a9-dbafba9cfe5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:21.826644 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b750759-ae93-43ab-81a9-dbafba9cfe5d None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:21.826924 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b750759-ae93-43ab-81a9-dbafba9cfe5d None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:21.827220 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b750759-ae93-43ab-81a9-dbafba9cfe5d None admin] PATH_INFO: `/v3/users/06a6d5cbc4324313a9f8745928cd5ca8` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:21.828439 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b750759-ae93-43ab-81a9-dbafba9cfe5d None admin] RBAC: Authorizing `identity:get_user(user_id=06a6d5cbc4324313a9f8745928cd5ca8)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:21.829900 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b750759-ae93-43ab-81a9-dbafba9cfe5d None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:21.831752 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 139/291] 10.222.0.26 () {56 vars in 1177 bytes} [Thu Jul 14 09:04:21 2022] GET /identity/v3/users/06a6d5cbc4324313a9f8745928cd5ca8 => generated 297 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:04:21.837410 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b21ecfc-5def-4b55-abff-de410b6cc7e5 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:22.062790 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b21ecfc-5def-4b55-abff-de410b6cc7e5 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:22.111805 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847f019f-f7d1-4520-bc63-d547f8cf7a8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:22.112387 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847f019f-f7d1-4520-bc63-d547f8cf7a8e None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:22.112669 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847f019f-f7d1-4520-bc63-d547f8cf7a8e None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:22.112946 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847f019f-f7d1-4520-bc63-d547f8cf7a8e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:22.136814 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:22.218652 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847f019f-f7d1-4520-bc63-d547f8cf7a8e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:22.220164 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847f019f-f7d1-4520-bc63-d547f8cf7a8e None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:22.357183 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 153/292] 10.222.0.26 () {58 vars in 1282 bytes} [Thu Jul 14 09:04:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 520 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 09:04:22.361799 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b750759-ae93-43ab-81a9-dbafba9cfe5d None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:22.523120 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b750759-ae93-43ab-81a9-dbafba9cfe5d None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:22.544305 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:22.583932 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889f7f84-2497-4e74-9bf7-21614e375c23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:22.584809 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889f7f84-2497-4e74-9bf7-21614e375c23 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:22.585094 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889f7f84-2497-4e74-9bf7-21614e375c23 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:22.585369 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889f7f84-2497-4e74-9bf7-21614e375c23 None admin] PATH_INFO: `/v3/projects/a596189576264bfc9638ca33f184759c` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:22.586587 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889f7f84-2497-4e74-9bf7-21614e375c23 None admin] RBAC: Authorizing `identity:get_project(project_id=a596189576264bfc9638ca33f184759c)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:22.588065 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889f7f84-2497-4e74-9bf7-21614e375c23 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:22.590092 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 140/293] 10.222.0.26 () {56 vars in 1186 bytes} [Thu Jul 14 09:04:22 2022] GET /identity/v3/projects/a596189576264bfc9638ca33f184759c => generated 302 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:04:22.595853 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847f019f-f7d1-4520-bc63-d547f8cf7a8e None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:22.651395 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e51384-4893-4993-83f5-bc919eab9852 None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:22.651939 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e51384-4893-4993-83f5-bc919eab9852 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:22.652357 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e51384-4893-4993-83f5-bc919eab9852 None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:22.653875 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 141/294] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:04:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 09:04:22.659062 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d238244b-12f8-426c-8d5f-a7a086094290 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:22.659539 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d238244b-12f8-426c-8d5f-a7a086094290 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:22.659958 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d238244b-12f8-426c-8d5f-a7a086094290 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:22.669234 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:22.704238 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-d238244b-12f8-426c-8d5f-a7a086094290 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:04:22.705446 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-d238244b-12f8-426c-8d5f-a7a086094290 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:04:22.723355 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847f019f-f7d1-4520-bc63-d547f8cf7a8e None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:22.759637 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bc3968-45b3-43c1-85b4-3f6cf15ec9e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:22.760843 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bc3968-45b3-43c1-85b4-3f6cf15ec9e1 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:22.761267 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bc3968-45b3-43c1-85b4-3f6cf15ec9e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:22.761680 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bc3968-45b3-43c1-85b4-3f6cf15ec9e1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:22.762650 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bc3968-45b3-43c1-85b4-3f6cf15ec9e1 None admin] RBAC: Adding query filter params (role.id=d1691322ce6a4edab7ce722edf0f2988, scope.project.id=a596189576264bfc9638ca33f184759c, user.id=06a6d5cbc4324313a9f8745928cd5ca8) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:04:22.763309 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bc3968-45b3-43c1-85b4-3f6cf15ec9e1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:22.764906 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bc3968-45b3-43c1-85b4-3f6cf15ec9e1 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:22.773350 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 154/295] 10.222.0.26 () {56 vars in 1374 bytes} [Thu Jul 14 09:04:22 2022] GET /identity/v3/role_assignments?user.id=06a6d5cbc4324313a9f8745928cd5ca8&scope.project.id=a596189576264bfc9638ca33f184759c&role.id=d1691322ce6a4edab7ce722edf0f2988 => generated 608 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:04:22.775163 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84085118-34cd-42ba-8b52-be9edbbcf40d None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:22.775650 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84085118-34cd-42ba-8b52-be9edbbcf40d None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:22.776045 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84085118-34cd-42ba-8b52-be9edbbcf40d None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:22.777454 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 155/296] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:04:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:04:22.785579 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b2d9f7-0e0d-4e64-9581-34c5d2456d8f None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:22.786083 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b2d9f7-0e0d-4e64-9581-34c5d2456d8f None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:22.786538 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b2d9f7-0e0d-4e64-9581-34c5d2456d8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:22.806857 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-90b2d9f7-0e0d-4e64-9581-34c5d2456d8f None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:04:22.808354 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-90b2d9f7-0e0d-4e64-9581-34c5d2456d8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:04:22.861826 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 142/297] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:04:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 09:04:22.891347 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 156/298] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:04:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 09:04:22.893655 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6840958c-ffa4-4d7c-b92d-a9cb0f019fa9 None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:22.894087 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6840958c-ffa4-4d7c-b92d-a9cb0f019fa9 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:22.894489 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6840958c-ffa4-4d7c-b92d-a9cb0f019fa9 None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:22.896593 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 143/299] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:04:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:04:22.897138 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b2d9f7-0e0d-4e64-9581-34c5d2456d8f None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:22.902169 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f993db-9486-40db-ad0f-2cb16d211013 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:22.902758 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f993db-9486-40db-ad0f-2cb16d211013 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:22.903262 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f993db-9486-40db-ad0f-2cb16d211013 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:22.925606 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-e8f993db-9486-40db-ad0f-2cb16d211013 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:04:22.926796 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-e8f993db-9486-40db-ad0f-2cb16d211013 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:04:22.966435 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:23.006014 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 144/300] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:04:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 09:04:23.006977 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f993db-9486-40db-ad0f-2cb16d211013 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:23.039767 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b2d9f7-0e0d-4e64-9581-34c5d2456d8f None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:23.053075 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:23.082109 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8591da35-16c7-46b8-a6d6-23eedcd929ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:23.083506 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8591da35-16c7-46b8-a6d6-23eedcd929ae None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:23.083958 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8591da35-16c7-46b8-a6d6-23eedcd929ae None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:23.084272 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8591da35-16c7-46b8-a6d6-23eedcd929ae None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:23.101788 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8591da35-16c7-46b8-a6d6-23eedcd929ae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 09:04:23.110240 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-8591da35-16c7-46b8-a6d6-23eedcd929ae None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 09:04:23.110240 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:04:23.110240 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:04:23.110240 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:04:23.110240 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:04:23.110240 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:04:23.110240 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:04:23.110240 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:04:23.110240 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:04:23.110240 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:04:23.110240 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:04:23.110240 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:04:23.110240 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 09:04:23.110240 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 09:04:23.110240 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 09:04:23.110240 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 09:04:23.111499 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 09:04:23.111499 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 09:04:23.111499 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 09:04:23.111499 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 09:04:23.111499 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:04:23.111499 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:04:23.111499 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:04:23.111499 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:04:23.111499 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:04:23.111499 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:04:23.111499 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:04:23.111499 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 14 09:04:23.111499 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:04:23.111499 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:04:23.111499 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:04:23.111499 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:04:23.112348 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:04:23.112348 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:04:23.112348 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:04:23.112348 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:04:23.112348 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 09:04:23.112348 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 09:04:23.112348 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 09:04:23.112348 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 09:04:23.112348 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 09:04:23.112348 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 09:04:23.112348 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 09:04:23.112348 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 14 09:04:23.112348 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 157/301] 10.222.0.26 () {56 vars in 1099 bytes} [Thu Jul 14 09:04:22 2022] GET /identity/v3/roles/reader => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 09:04:23.113053 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8591da35-16c7-46b8-a6d6-23eedcd929ae None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:23.125951 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f993db-9486-40db-ad0f-2cb16d211013 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:23.160604 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187565f3-f851-483c-98bc-9951fd4eecb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:23.161500 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187565f3-f851-483c-98bc-9951fd4eecb2 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:23.161789 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187565f3-f851-483c-98bc-9951fd4eecb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:23.162065 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187565f3-f851-483c-98bc-9951fd4eecb2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:23.172391 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187565f3-f851-483c-98bc-9951fd4eecb2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 09:04:23.180508 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-187565f3-f851-483c-98bc-9951fd4eecb2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 09:04:23.180508 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:04:23.180508 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:04:23.180508 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:04:23.180508 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:04:23.180508 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:04:23.180508 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:04:23.180508 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:04:23.180508 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:04:23.180508 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:04:23.180508 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:04:23.180508 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:04:23.180508 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 09:04:23.180508 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 09:04:23.180508 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 09:04:23.180508 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 09:04:23.181554 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 09:04:23.181554 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 09:04:23.181554 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 09:04:23.181554 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 09:04:23.181554 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:04:23.181554 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:04:23.181554 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:04:23.181554 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:04:23.181554 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:04:23.181554 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:04:23.181554 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:04:23.181554 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 14 09:04:23.181554 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:04:23.181554 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:04:23.181554 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:04:23.181554 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:04:23.182371 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:04:23.182371 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:04:23.182371 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:04:23.182371 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:04:23.182371 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 09:04:23.182371 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 09:04:23.182371 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 09:04:23.182371 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 09:04:23.182371 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 09:04:23.182371 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 09:04:23.182371 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 09:04:23.182371 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 14 09:04:23.182371 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 145/302] 10.222.0.26 () {56 vars in 1099 bytes} [Thu Jul 14 09:04:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:04:23.183032 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187565f3-f851-483c-98bc-9951fd4eecb2 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:23.227436 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:23.231420 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8591da35-16c7-46b8-a6d6-23eedcd929ae None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:23.269302 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3ef25b-a83f-4a47-a1f0-555c5d3d858b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:23.270420 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3ef25b-a83f-4a47-a1f0-555c5d3d858b None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:23.270751 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3ef25b-a83f-4a47-a1f0-555c5d3d858b None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:23.271058 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3ef25b-a83f-4a47-a1f0-555c5d3d858b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:23.272335 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3ef25b-a83f-4a47-a1f0-555c5d3d858b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:23.273804 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3ef25b-a83f-4a47-a1f0-555c5d3d858b None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:23.275645 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 158/303] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jul 14 09:04:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:04:23.280705 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3ef25b-a83f-4a47-a1f0-555c5d3d858b None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:23.287939 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187565f3-f851-483c-98bc-9951fd4eecb2 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:23.289642 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:23.329497 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6218aa1-d67c-4de9-ad89-669a936127eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:23.330374 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6218aa1-d67c-4de9-ad89-669a936127eb None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:23.330854 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6218aa1-d67c-4de9-ad89-669a936127eb None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:23.331236 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6218aa1-d67c-4de9-ad89-669a936127eb None admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:23.332065 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6218aa1-d67c-4de9-ad89-669a936127eb None admin] RBAC: Adding query filter params (name=reader) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:04:23.332420 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6218aa1-d67c-4de9-ad89-669a936127eb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:23.334343 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6218aa1-d67c-4de9-ad89-669a936127eb None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:23.345356 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 146/304] 10.222.0.26 () {56 vars in 1101 bytes} [Thu Jul 14 09:04:23 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:04:23.346641 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6218aa1-d67c-4de9-ad89-669a936127eb None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:23.401066 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3ef25b-a83f-4a47-a1f0-555c5d3d858b None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:23.434242 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272e1850-c799-41c8-a876-846a5a84d0a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:23.435063 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272e1850-c799-41c8-a876-846a5a84d0a4 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:23.435378 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272e1850-c799-41c8-a876-846a5a84d0a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:23.435659 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272e1850-c799-41c8-a876-846a5a84d0a4 None admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:23.436451 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272e1850-c799-41c8-a876-846a5a84d0a4 None admin] RBAC: Adding query filter params (name=member) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:04:23.436780 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272e1850-c799-41c8-a876-846a5a84d0a4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:23.438180 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272e1850-c799-41c8-a876-846a5a84d0a4 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:23.446683 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 159/305] 10.222.0.26 () {56 vars in 1101 bytes} [Thu Jul 14 09:04:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:04:23.446683 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272e1850-c799-41c8-a876-846a5a84d0a4 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:23.453762 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6218aa1-d67c-4de9-ad89-669a936127eb None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:23.480081 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:23.488613 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b082c35-0732-49f6-bbac-cd6c4a54be84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:23.489310 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b082c35-0732-49f6-bbac-cd6c4a54be84 None admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:23.489624 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b082c35-0732-49f6-bbac-cd6c4a54be84 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:23.489913 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b082c35-0732-49f6-bbac-cd6c4a54be84 None admin] PATH_INFO: `/v3/groups` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:23.490597 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b082c35-0732-49f6-bbac-cd6c4a54be84 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:23.492104 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b082c35-0732-49f6-bbac-cd6c4a54be84 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:23.507368 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 147/306] 10.222.0.26 () {60 vars in 1134 bytes} [Thu Jul 14 09:04:23 2022] POST /identity/v3/groups => generated 229 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 09:04:23.508872 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e20676-259e-46ca-ba38-cfd5b68614d2 None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:23.509162 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e20676-259e-46ca-ba38-cfd5b68614d2 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:23.509440 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e20676-259e-46ca-ba38-cfd5b68614d2 None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:23.510875 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 148/307] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:04:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:04:23.511470 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e20676-259e-46ca-ba38-cfd5b68614d2 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:23.545453 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:23.570480 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272e1850-c799-41c8-a876-846a5a84d0a4 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:23.615967 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2658823b-1d93-4893-93dd-a855b3dff224 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:23.616920 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2658823b-1d93-4893-93dd-a855b3dff224 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:23.617306 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2658823b-1d93-4893-93dd-a855b3dff224 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:23.617488 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2658823b-1d93-4893-93dd-a855b3dff224 None admin] PATH_INFO: `/v3/users/30bb7e4926d944aeb839cc90e9c8ab7f` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:23.632414 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2658823b-1d93-4893-93dd-a855b3dff224 None admin] RBAC: Authorizing `identity:get_user(user_id=30bb7e4926d944aeb839cc90e9c8ab7f)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:23.633908 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2658823b-1d93-4893-93dd-a855b3dff224 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:23.640463 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 160/308] 10.222.0.26 () {56 vars in 1177 bytes} [Thu Jul 14 09:04:23 2022] GET /identity/v3/users/30bb7e4926d944aeb839cc90e9c8ab7f => generated 293 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:04:23.640463 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc28c99-9d64-4d40-9d83-fa3c6fc0cda7 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:23.640463 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc28c99-9d64-4d40-9d83-fa3c6fc0cda7 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:23.640463 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc28c99-9d64-4d40-9d83-fa3c6fc0cda7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:23.649643 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e20676-259e-46ca-ba38-cfd5b68614d2 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:23.654946 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-5cc28c99-9d64-4d40-9d83-fa3c6fc0cda7 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:04:23.656211 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-5cc28c99-9d64-4d40-9d83-fa3c6fc0cda7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:04:23.691407 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac16164-8e44-4881-952d-f0f18149f704 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:23.692647 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac16164-8e44-4881-952d-f0f18149f704 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:23.692956 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac16164-8e44-4881-952d-f0f18149f704 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:23.693251 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac16164-8e44-4881-952d-f0f18149f704 None admin] PATH_INFO: `/v3/users/617dad19c58948b38a5f12260c00d9af` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:23.694505 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac16164-8e44-4881-952d-f0f18149f704 None admin] RBAC: Authorizing `identity:get_user(user_id=617dad19c58948b38a5f12260c00d9af)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:23.695978 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac16164-8e44-4881-952d-f0f18149f704 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:23.697710 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 149/309] 10.222.0.26 () {56 vars in 1177 bytes} [Thu Jul 14 09:04:23 2022] GET /identity/v3/users/617dad19c58948b38a5f12260c00d9af => generated 293 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:04:23.698562 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac16164-8e44-4881-952d-f0f18149f704 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:23.732243 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 161/310] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:04:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 09:04:23.733534 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc28c99-9d64-4d40-9d83-fa3c6fc0cda7 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:23.753488 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:23.794978 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:23.805722 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac16164-8e44-4881-952d-f0f18149f704 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:23.839927 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc28c99-9d64-4d40-9d83-fa3c6fc0cda7 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:23.840122 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5361ccc-01a3-44e4-ab7c-65503a677463 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:23.840648 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5361ccc-01a3-44e4-ab7c-65503a677463 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:23.840927 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5361ccc-01a3-44e4-ab7c-65503a677463 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:23.841204 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5361ccc-01a3-44e4-ab7c-65503a677463 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:23.842050 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5361ccc-01a3-44e4-ab7c-65503a677463 None admin] RBAC: Adding query filter params (role.id=9c412baac5914ce6a40ad7d3f0f0a0d6, user.id=30bb7e4926d944aeb839cc90e9c8ab7f, scope.system=all) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:04:23.842366 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5361ccc-01a3-44e4-ab7c-65503a677463 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:23.843885 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5361ccc-01a3-44e4-ab7c-65503a677463 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:23.859232 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 150/311] 10.222.0.26 () {56 vars in 1308 bytes} [Thu Jul 14 09:04:23 2022] GET /identity/v3/role_assignments?user.id=30bb7e4926d944aeb839cc90e9c8ab7f&scope.system=all&role.id=9c412baac5914ce6a40ad7d3f0f0a0d6 => generated 226 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:04:23.859232 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5361ccc-01a3-44e4-ab7c-65503a677463 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:23.906763 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4e8a22-2348-46cf-afc6-068ede1ed5c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:23.907737 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4e8a22-2348-46cf-afc6-068ede1ed5c4 None admin] REQUEST_METHOD: `PUT` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:23.908087 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4e8a22-2348-46cf-afc6-068ede1ed5c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:23.908429 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4e8a22-2348-46cf-afc6-068ede1ed5c4 None admin] PATH_INFO: `/v3/system/users/617dad19c58948b38a5f12260c00d9af/roles/d1691322ce6a4edab7ce722edf0f2988` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:23.910209 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4e8a22-2348-46cf-afc6-068ede1ed5c4 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=617dad19c58948b38a5f12260c00d9af, role_id=d1691322ce6a4edab7ce722edf0f2988)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:23.911853 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4e8a22-2348-46cf-afc6-068ede1ed5c4 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:23.925086 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 162/312] 10.222.0.26 () {58 vars in 1334 bytes} [Thu Jul 14 09:04:23 2022] PUT /identity/v3/system/users/617dad19c58948b38a5f12260c00d9af/roles/d1691322ce6a4edab7ce722edf0f2988 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:04:23.994645 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5361ccc-01a3-44e4-ab7c-65503a677463 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:24.038139 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7409a1f-fbf4-4836-8660-17fba32b0ac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:24.039014 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7409a1f-fbf4-4836-8660-17fba32b0ac7 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:24.039337 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7409a1f-fbf4-4836-8660-17fba32b0ac7 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:24.039619 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7409a1f-fbf4-4836-8660-17fba32b0ac7 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:24.051753 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7409a1f-fbf4-4836-8660-17fba32b0ac7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 09:04:24.060040 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-d7409a1f-fbf4-4836-8660-17fba32b0ac7 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 09:04:24.060040 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:04:24.060040 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:04:24.060040 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:04:24.060040 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:04:24.060040 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:04:24.060040 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:04:24.060040 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:04:24.060040 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:04:24.060040 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:04:24.060040 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:04:24.060040 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:04:24.060040 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 09:04:24.060040 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 09:04:24.060040 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 09:04:24.060040 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 09:04:24.061004 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 09:04:24.061004 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 09:04:24.061004 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 09:04:24.061004 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 09:04:24.061004 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:04:24.061004 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:04:24.061004 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:04:24.061004 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:04:24.061004 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:04:24.061004 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:04:24.061004 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:04:24.061004 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 14 09:04:24.061004 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:04:24.061004 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:04:24.061004 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:04:24.061004 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:04:24.062911 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:04:24.062911 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:04:24.062911 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:04:24.062911 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:04:24.062911 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 09:04:24.062911 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 09:04:24.062911 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 09:04:24.062911 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 09:04:24.062911 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 09:04:24.062911 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 09:04:24.062911 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 09:04:24.062911 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 14 09:04:24.062911 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 151/313] 10.222.0.26 () {56 vars in 1099 bytes} [Thu Jul 14 09:04:23 2022] GET /identity/v3/roles/reader => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:04:24.068428 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4e8a22-2348-46cf-afc6-068ede1ed5c4 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:24.166561 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:24.187129 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4e8a22-2348-46cf-afc6-068ede1ed5c4 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:24.222284 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ede71c-83db-4308-8c0c-9e1ce973a5ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:24.223316 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ede71c-83db-4308-8c0c-9e1ce973a5ac None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:24.223806 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ede71c-83db-4308-8c0c-9e1ce973a5ac None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:24.224255 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ede71c-83db-4308-8c0c-9e1ce973a5ac None admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:24.225219 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ede71c-83db-4308-8c0c-9e1ce973a5ac None admin] RBAC: Adding query filter params (name=reader) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:04:24.225698 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ede71c-83db-4308-8c0c-9e1ce973a5ac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:24.227444 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ede71c-83db-4308-8c0c-9e1ce973a5ac None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:24.236797 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 163/314] 10.222.0.26 () {56 vars in 1101 bytes} [Thu Jul 14 09:04:24 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:04:24.242759 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7409a1f-fbf4-4836-8660-17fba32b0ac7 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:24.262718 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:24.352053 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7409a1f-fbf4-4836-8660-17fba32b0ac7 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:24.392432 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8b4626-3072-42aa-98e3-5459fe101526 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:24.393663 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8b4626-3072-42aa-98e3-5459fe101526 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:24.394145 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8b4626-3072-42aa-98e3-5459fe101526 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:24.394645 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8b4626-3072-42aa-98e3-5459fe101526 None admin] PATH_INFO: `/v3/users/d7e17cf35330491aa753de3b5df8db3d` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:24.396167 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8b4626-3072-42aa-98e3-5459fe101526 None admin] RBAC: Authorizing `identity:get_user(user_id=d7e17cf35330491aa753de3b5df8db3d)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:24.397831 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8b4626-3072-42aa-98e3-5459fe101526 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:24.404244 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 152/315] 10.222.0.26 () {56 vars in 1177 bytes} [Thu Jul 14 09:04:24 2022] GET /identity/v3/users/d7e17cf35330491aa753de3b5df8db3d => generated 297 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:04:24.405802 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ede71c-83db-4308-8c0c-9e1ce973a5ac None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:24.514993 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ede71c-83db-4308-8c0c-9e1ce973a5ac None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:24.551367 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b62be5-5818-4294-9cce-c243497b967f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:24.552454 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b62be5-5818-4294-9cce-c243497b967f None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:24.559210 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b62be5-5818-4294-9cce-c243497b967f None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:24.559210 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b62be5-5818-4294-9cce-c243497b967f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:24.568299 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:24.623588 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b62be5-5818-4294-9cce-c243497b967f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:24.625062 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b62be5-5818-4294-9cce-c243497b967f None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:24.727787 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 164/316] 10.222.0.26 () {58 vars in 1282 bytes} [Thu Jul 14 09:04:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 09:04:24.734451 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8b4626-3072-42aa-98e3-5459fe101526 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:24.823742 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:24.840009 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8b4626-3072-42aa-98e3-5459fe101526 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:24.874515 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304bc998-3117-428e-96b5-b74bb2fef1f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:24.875408 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304bc998-3117-428e-96b5-b74bb2fef1f9 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:24.875696 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304bc998-3117-428e-96b5-b74bb2fef1f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:24.875974 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304bc998-3117-428e-96b5-b74bb2fef1f9 None admin] PATH_INFO: `/v3/projects/a596189576264bfc9638ca33f184759c` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:24.877175 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304bc998-3117-428e-96b5-b74bb2fef1f9 None admin] RBAC: Authorizing `identity:get_project(project_id=a596189576264bfc9638ca33f184759c)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:24.878645 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304bc998-3117-428e-96b5-b74bb2fef1f9 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:24.885389 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 153/317] 10.222.0.26 () {56 vars in 1186 bytes} [Thu Jul 14 09:04:24 2022] GET /identity/v3/projects/a596189576264bfc9638ca33f184759c => generated 302 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:04:24.886753 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b62be5-5818-4294-9cce-c243497b967f None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:24.932239 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:24.985827 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b62be5-5818-4294-9cce-c243497b967f None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:25.022012 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5b3117-d65e-425c-aeef-16824f1995e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:25.022683 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5b3117-d65e-425c-aeef-16824f1995e4 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:25.022960 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5b3117-d65e-425c-aeef-16824f1995e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:25.023264 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5b3117-d65e-425c-aeef-16824f1995e4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:25.024111 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5b3117-d65e-425c-aeef-16824f1995e4 None admin] RBAC: Adding query filter params (role.id=9c412baac5914ce6a40ad7d3f0f0a0d6, scope.project.id=a596189576264bfc9638ca33f184759c, user.id=d7e17cf35330491aa753de3b5df8db3d) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:04:25.024422 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5b3117-d65e-425c-aeef-16824f1995e4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:25.025865 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5b3117-d65e-425c-aeef-16824f1995e4 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:25.032376 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 165/318] 10.222.0.26 () {56 vars in 1374 bytes} [Thu Jul 14 09:04:24 2022] GET /identity/v3/role_assignments?user.id=d7e17cf35330491aa753de3b5df8db3d&scope.project.id=a596189576264bfc9638ca33f184759c&role.id=9c412baac5914ce6a40ad7d3f0f0a0d6 => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 09:04:25.598798 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0eb9069-da3e-49e4-9a95-320596583ccc None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:25.599122 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0eb9069-da3e-49e4-9a95-320596583ccc None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:25.599411 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0eb9069-da3e-49e4-9a95-320596583ccc None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:25.600841 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 154/319] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:04:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:04:25.606017 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15022988-1cd8-4fd5-b2fa-64b80da1e2c8 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:25.606256 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15022988-1cd8-4fd5-b2fa-64b80da1e2c8 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:25.606553 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15022988-1cd8-4fd5-b2fa-64b80da1e2c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:25.625879 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-15022988-1cd8-4fd5-b2fa-64b80da1e2c8 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:04:25.627051 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-15022988-1cd8-4fd5-b2fa-64b80da1e2c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:04:25.698210 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 166/320] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:04:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 09:04:25.720189 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0eb9069-da3e-49e4-9a95-320596583ccc None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:25.829458 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0eb9069-da3e-49e4-9a95-320596583ccc None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:25.863696 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45d0a55-4c40-4a3e-b409-bf6b26d27b2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:25.864525 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45d0a55-4c40-4a3e-b409-bf6b26d27b2a None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:25.864809 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45d0a55-4c40-4a3e-b409-bf6b26d27b2a None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:25.865084 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45d0a55-4c40-4a3e-b409-bf6b26d27b2a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:25.866253 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45d0a55-4c40-4a3e-b409-bf6b26d27b2a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:25.867760 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45d0a55-4c40-4a3e-b409-bf6b26d27b2a None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:25.869561 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 155/321] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jul 14 09:04:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:04:25.875149 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15022988-1cd8-4fd5-b2fa-64b80da1e2c8 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:25.914926 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:25.921006 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5880d6f-ef96-490e-8164-33633559f91a None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:25.921283 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5880d6f-ef96-490e-8164-33633559f91a None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:25.921571 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5880d6f-ef96-490e-8164-33633559f91a None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:25.923094 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 156/322] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:04:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:04:25.929055 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0129a0-22e2-4124-94f8-01f0f1cffd0e None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:25.929341 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0129a0-22e2-4124-94f8-01f0f1cffd0e None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:25.929654 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0129a0-22e2-4124-94f8-01f0f1cffd0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:25.946075 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-0b0129a0-22e2-4124-94f8-01f0f1cffd0e None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:04:25.947303 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-0b0129a0-22e2-4124-94f8-01f0f1cffd0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:04:25.962116 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:25.983800 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15022988-1cd8-4fd5-b2fa-64b80da1e2c8 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:26.020537 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeaca890-857c-4efc-8a05-c5b0bd6f32d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:26.021210 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaca890-857c-4efc-8a05-c5b0bd6f32d1 None admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:26.021491 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaca890-857c-4efc-8a05-c5b0bd6f32d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:26.021753 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaca890-857c-4efc-8a05-c5b0bd6f32d1 None admin] PATH_INFO: `/v3/groups` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:26.022449 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeaca890-857c-4efc-8a05-c5b0bd6f32d1 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:26.023932 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeaca890-857c-4efc-8a05-c5b0bd6f32d1 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:26.026587 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 157/323] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:04:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 09:04:26.041675 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 167/324] 10.222.0.26 () {60 vars in 1134 bytes} [Thu Jul 14 09:04:25 2022] POST /identity/v3/groups => generated 226 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 09:04:26.051183 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0129a0-22e2-4124-94f8-01f0f1cffd0e None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:26.081113 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae05dbd-6701-488d-91a5-0610e2a384ae None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:26.081288 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae05dbd-6701-488d-91a5-0610e2a384ae None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:26.081547 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae05dbd-6701-488d-91a5-0610e2a384ae None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:26.083261 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 168/325] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:04:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 09:04:26.088576 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1b6014-324c-4476-a371-078ecfcf86c7 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:26.088894 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1b6014-324c-4476-a371-078ecfcf86c7 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:26.089141 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1b6014-324c-4476-a371-078ecfcf86c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:26.106637 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-9c1b6014-324c-4476-a371-078ecfcf86c7 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:04:26.107961 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-9c1b6014-324c-4476-a371-078ecfcf86c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:04:26.177407 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0129a0-22e2-4124-94f8-01f0f1cffd0e None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:26.182005 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 169/326] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:04:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 09:04:26.204771 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1b6014-324c-4476-a371-078ecfcf86c7 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:26.220893 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ca88c4-aa4c-46d7-9493-78c022f5520a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:26.221899 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ca88c4-aa4c-46d7-9493-78c022f5520a None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:26.222586 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ca88c4-aa4c-46d7-9493-78c022f5520a None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:26.222842 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ca88c4-aa4c-46d7-9493-78c022f5520a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:26.234555 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ca88c4-aa4c-46d7-9493-78c022f5520a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 09:04:26.236211 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:26.242473 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-39ca88c4-aa4c-46d7-9493-78c022f5520a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 09:04:26.242473 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:04:26.242473 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:04:26.242473 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:04:26.242473 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:04:26.242473 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:04:26.242473 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:04:26.242473 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:04:26.242473 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:04:26.242473 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:04:26.242473 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:04:26.242473 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:04:26.242473 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 09:04:26.242473 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 09:04:26.242473 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 09:04:26.242473 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 09:04:26.243672 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 09:04:26.243672 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 09:04:26.243672 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 09:04:26.243672 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 09:04:26.243672 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:04:26.243672 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:04:26.243672 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:04:26.243672 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:04:26.243672 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:04:26.243672 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:04:26.243672 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:04:26.243672 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 14 09:04:26.243672 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:04:26.243672 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:04:26.243672 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:04:26.243672 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:04:26.244637 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:04:26.244637 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:04:26.244637 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:04:26.244637 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:04:26.244637 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 09:04:26.244637 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 09:04:26.244637 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 09:04:26.244637 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 09:04:26.244637 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 09:04:26.244637 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 09:04:26.244637 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 09:04:26.244637 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 14 09:04:26.244637 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 158/327] 10.222.0.26 () {56 vars in 1099 bytes} [Thu Jul 14 09:04:26 2022] GET /identity/v3/roles/reader => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 09:04:26.249520 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ca88c4-aa4c-46d7-9493-78c022f5520a None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:26.265221 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:26.325176 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1b6014-324c-4476-a371-078ecfcf86c7 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:26.369326 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ca88c4-aa4c-46d7-9493-78c022f5520a None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:26.369499 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fa755b-4a23-4c36-8cc3-d00eaa09620c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:26.370069 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fa755b-4a23-4c36-8cc3-d00eaa09620c None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:26.370354 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fa755b-4a23-4c36-8cc3-d00eaa09620c None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:26.370737 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fa755b-4a23-4c36-8cc3-d00eaa09620c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:26.382901 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fa755b-4a23-4c36-8cc3-d00eaa09620c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 09:04:26.390733 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-b1fa755b-4a23-4c36-8cc3-d00eaa09620c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 09:04:26.390733 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:04:26.390733 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:04:26.390733 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:04:26.390733 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:04:26.390733 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:04:26.390733 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:04:26.390733 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:04:26.390733 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:04:26.390733 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:04:26.390733 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:04:26.390733 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:04:26.390733 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 09:04:26.390733 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 09:04:26.390733 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 09:04:26.390733 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 09:04:26.391677 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 09:04:26.391677 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 09:04:26.391677 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 09:04:26.391677 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 09:04:26.391677 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:04:26.391677 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:04:26.391677 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:04:26.391677 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:04:26.391677 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:04:26.391677 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:04:26.391677 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:04:26.391677 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 14 09:04:26.391677 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:04:26.391677 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:04:26.391677 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:04:26.391677 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:04:26.392489 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:04:26.392489 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:04:26.392489 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:04:26.392489 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:04:26.392489 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 09:04:26.392489 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 09:04:26.392489 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 09:04:26.392489 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 09:04:26.392489 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 09:04:26.392489 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 09:04:26.392489 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 09:04:26.392489 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 14 09:04:26.392489 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 170/328] 10.222.0.26 () {56 vars in 1099 bytes} [Thu Jul 14 09:04:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:04:26.400655 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fa755b-4a23-4c36-8cc3-d00eaa09620c None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:26.411310 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027f5601-23a5-438d-9d47-a87ed8357dcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:26.412114 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027f5601-23a5-438d-9d47-a87ed8357dcb None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:26.414381 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027f5601-23a5-438d-9d47-a87ed8357dcb None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:26.414557 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027f5601-23a5-438d-9d47-a87ed8357dcb None admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:26.415445 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027f5601-23a5-438d-9d47-a87ed8357dcb None admin] RBAC: Adding query filter params (name=reader) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:04:26.418471 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027f5601-23a5-438d-9d47-a87ed8357dcb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:26.418471 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027f5601-23a5-438d-9d47-a87ed8357dcb None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:26.429518 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 159/329] 10.222.0.26 () {56 vars in 1101 bytes} [Thu Jul 14 09:04:26 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:04:26.438870 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027f5601-23a5-438d-9d47-a87ed8357dcb None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:26.511668 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:26.526691 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:26.526893 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fa755b-4a23-4c36-8cc3-d00eaa09620c None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:26.559366 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027f5601-23a5-438d-9d47-a87ed8357dcb None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:26.562514 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c22e6b0-415a-47ed-a3ef-f17d6968f9b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:26.563239 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c22e6b0-415a-47ed-a3ef-f17d6968f9b7 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:26.563855 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c22e6b0-415a-47ed-a3ef-f17d6968f9b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:26.564146 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c22e6b0-415a-47ed-a3ef-f17d6968f9b7 None admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:26.564919 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c22e6b0-415a-47ed-a3ef-f17d6968f9b7 None admin] RBAC: Adding query filter params (name=member) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:04:26.565232 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c22e6b0-415a-47ed-a3ef-f17d6968f9b7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:26.566696 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c22e6b0-415a-47ed-a3ef-f17d6968f9b7 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:26.576315 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 171/330] 10.222.0.26 () {56 vars in 1101 bytes} [Thu Jul 14 09:04:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:04:26.581804 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c22e6b0-415a-47ed-a3ef-f17d6968f9b7 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:26.595122 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf58fa87-6b34-4221-b3d3-bdfdb1bba3db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:26.596079 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf58fa87-6b34-4221-b3d3-bdfdb1bba3db None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:26.596362 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf58fa87-6b34-4221-b3d3-bdfdb1bba3db None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:26.596633 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf58fa87-6b34-4221-b3d3-bdfdb1bba3db None admin] PATH_INFO: `/v3/users/30bb7e4926d944aeb839cc90e9c8ab7f` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:26.597929 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf58fa87-6b34-4221-b3d3-bdfdb1bba3db None admin] RBAC: Authorizing `identity:get_user(user_id=30bb7e4926d944aeb839cc90e9c8ab7f)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:26.600233 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf58fa87-6b34-4221-b3d3-bdfdb1bba3db None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:26.602241 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 160/331] 10.222.0.26 () {56 vars in 1177 bytes} [Thu Jul 14 09:04:26 2022] GET /identity/v3/users/30bb7e4926d944aeb839cc90e9c8ab7f => generated 293 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 09:04:26.607665 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf58fa87-6b34-4221-b3d3-bdfdb1bba3db None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:26.686877 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c22e6b0-415a-47ed-a3ef-f17d6968f9b7 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:26.712428 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf58fa87-6b34-4221-b3d3-bdfdb1bba3db None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:26.721455 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea05992-0934-455a-b081-3cd888435a76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:26.722421 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea05992-0934-455a-b081-3cd888435a76 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:26.722891 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea05992-0934-455a-b081-3cd888435a76 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:26.723162 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea05992-0934-455a-b081-3cd888435a76 None admin] PATH_INFO: `/v3/users/617dad19c58948b38a5f12260c00d9af` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:26.724485 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea05992-0934-455a-b081-3cd888435a76 None admin] RBAC: Authorizing `identity:get_user(user_id=617dad19c58948b38a5f12260c00d9af)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:26.726006 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea05992-0934-455a-b081-3cd888435a76 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:26.728789 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 172/332] 10.222.0.26 () {56 vars in 1177 bytes} [Thu Jul 14 09:04:26 2022] GET /identity/v3/users/617dad19c58948b38a5f12260c00d9af => generated 293 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 09:04:26.734604 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea05992-0934-455a-b081-3cd888435a76 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:26.747817 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2088458d-faa1-4271-8e47-edb7c055c3ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:26.748683 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2088458d-faa1-4271-8e47-edb7c055c3ef None admin] REQUEST_METHOD: `PUT` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:26.748966 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2088458d-faa1-4271-8e47-edb7c055c3ef None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:26.749238 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2088458d-faa1-4271-8e47-edb7c055c3ef None admin] PATH_INFO: `/v3/system/users/30bb7e4926d944aeb839cc90e9c8ab7f/roles/9c412baac5914ce6a40ad7d3f0f0a0d6` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:26.750723 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2088458d-faa1-4271-8e47-edb7c055c3ef None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=30bb7e4926d944aeb839cc90e9c8ab7f, role_id=9c412baac5914ce6a40ad7d3f0f0a0d6)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:26.752142 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2088458d-faa1-4271-8e47-edb7c055c3ef None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:26.758304 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:26.768588 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 161/333] 10.222.0.26 () {58 vars in 1334 bytes} [Thu Jul 14 09:04:26 2022] PUT /identity/v3/system/users/30bb7e4926d944aeb839cc90e9c8ab7f/roles/9c412baac5914ce6a40ad7d3f0f0a0d6 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 09:04:26.856148 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea05992-0934-455a-b081-3cd888435a76 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:26.893069 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4e098b-e8b8-44fe-8a2b-f6b10a299d6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:26.893742 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4e098b-e8b8-44fe-8a2b-f6b10a299d6b None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:26.894021 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4e098b-e8b8-44fe-8a2b-f6b10a299d6b None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:26.894284 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4e098b-e8b8-44fe-8a2b-f6b10a299d6b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:26.895184 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4e098b-e8b8-44fe-8a2b-f6b10a299d6b None admin] RBAC: Adding query filter params (role.id=d1691322ce6a4edab7ce722edf0f2988, user.id=617dad19c58948b38a5f12260c00d9af, scope.system=all) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:04:26.895516 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4e098b-e8b8-44fe-8a2b-f6b10a299d6b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:26.896967 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4e098b-e8b8-44fe-8a2b-f6b10a299d6b None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:26.911061 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 173/334] 10.222.0.26 () {56 vars in 1308 bytes} [Thu Jul 14 09:04:26 2022] GET /identity/v3/role_assignments?user.id=617dad19c58948b38a5f12260c00d9af&scope.system=all&role.id=d1691322ce6a4edab7ce722edf0f2988 => generated 510 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 09:04:28.375582 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0a9f9a-9454-4d5e-a04f-073906108825 None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:28.376131 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0a9f9a-9454-4d5e-a04f-073906108825 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:28.376131 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0a9f9a-9454-4d5e-a04f-073906108825 None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:28.377785 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 162/335] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:04:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:04:28.383351 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a567614-2287-439a-88c3-b3e67b7b69fa None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:28.384195 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a567614-2287-439a-88c3-b3e67b7b69fa None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:28.384661 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a567614-2287-439a-88c3-b3e67b7b69fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:28.402468 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-1a567614-2287-439a-88c3-b3e67b7b69fa None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:04:28.403986 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-1a567614-2287-439a-88c3-b3e67b7b69fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:04:28.485804 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 174/336] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:04:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 09:04:28.504819 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeace090-77a7-4361-8e8e-7f99d01ec07c None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:28.505449 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeace090-77a7-4361-8e8e-7f99d01ec07c None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:28.505913 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeace090-77a7-4361-8e8e-7f99d01ec07c None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:28.507741 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 163/337] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:04:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:04:28.513422 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf5ae7a-dabd-4fa7-a6df-0407de080e8c None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:28.513708 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf5ae7a-dabd-4fa7-a6df-0407de080e8c None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:28.513982 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf5ae7a-dabd-4fa7-a6df-0407de080e8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:28.525406 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:28.532055 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeace090-77a7-4361-8e8e-7f99d01ec07c None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:28.536443 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-ecf5ae7a-dabd-4fa7-a6df-0407de080e8c None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:04:28.538616 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:28.538731 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-ecf5ae7a-dabd-4fa7-a6df-0407de080e8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:04:28.671483 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 175/338] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:04:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 09:04:28.709122 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf5ae7a-dabd-4fa7-a6df-0407de080e8c None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:28.738408 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeace090-77a7-4361-8e8e-7f99d01ec07c None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:28.818473 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464b7f3a-a344-4c98-8b06-92faa5a230a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:28.819298 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464b7f3a-a344-4c98-8b06-92faa5a230a9 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:28.819631 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464b7f3a-a344-4c98-8b06-92faa5a230a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:28.819900 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464b7f3a-a344-4c98-8b06-92faa5a230a9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:28.840309 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464b7f3a-a344-4c98-8b06-92faa5a230a9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 09:04:28.851103 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-464b7f3a-a344-4c98-8b06-92faa5a230a9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 09:04:28.851103 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:04:28.851103 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:04:28.851103 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:04:28.851103 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:04:28.851103 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:04:28.851103 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:04:28.851103 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:04:28.851103 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:04:28.851103 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:04:28.851103 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:04:28.851103 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:04:28.851103 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 09:04:28.851103 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 09:04:28.851103 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 09:04:28.851103 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 09:04:28.852037 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 09:04:28.852037 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 09:04:28.852037 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 09:04:28.852037 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 09:04:28.852037 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:04:28.852037 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:04:28.852037 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:04:28.852037 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:04:28.852037 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:04:28.852037 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:04:28.852037 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:04:28.852037 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 14 09:04:28.852037 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:04:28.852037 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:04:28.852037 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:04:28.852037 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:04:28.852833 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:04:28.852833 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:04:28.852833 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:04:28.852833 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:04:28.852833 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 09:04:28.852833 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 09:04:28.852833 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 09:04:28.852833 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 09:04:28.852833 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 09:04:28.852833 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 09:04:28.852833 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 09:04:28.852833 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 14 09:04:28.854996 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 164/339] 10.222.0.26 () {56 vars in 1099 bytes} [Thu Jul 14 09:04:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:04:28.861847 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464b7f3a-a344-4c98-8b06-92faa5a230a9 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:28.926545 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf5ae7a-dabd-4fa7-a6df-0407de080e8c None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:28.967215 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:28.973070 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c8984e-6814-4727-95cb-ee4788ea352b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:28.973911 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c8984e-6814-4727-95cb-ee4788ea352b None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:28.978673 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c8984e-6814-4727-95cb-ee4788ea352b None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:28.978958 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c8984e-6814-4727-95cb-ee4788ea352b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:28.982437 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:28.999235 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c8984e-6814-4727-95cb-ee4788ea352b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 09:04:29.007421 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-a3c8984e-6814-4727-95cb-ee4788ea352b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 09:04:29.007421 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:04:29.007421 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:04:29.007421 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:04:29.007421 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:04:29.007421 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:04:29.007421 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:04:29.007421 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:04:29.007421 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:04:29.007421 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:04:29.007421 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:04:29.007421 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:04:29.007421 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 09:04:29.007421 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 09:04:29.007421 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 09:04:29.007421 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 09:04:29.008301 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 09:04:29.008301 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 09:04:29.008301 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 09:04:29.008301 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 09:04:29.008301 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:04:29.008301 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:04:29.008301 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:04:29.008301 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:04:29.008301 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:04:29.008301 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:04:29.008301 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:04:29.008301 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 14 09:04:29.008301 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:04:29.008301 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:04:29.008301 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:04:29.008301 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:04:29.009526 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:04:29.009526 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:04:29.009526 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:04:29.009526 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:04:29.009526 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 09:04:29.009526 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 09:04:29.009526 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 09:04:29.009526 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 09:04:29.009526 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 09:04:29.009526 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 09:04:29.009526 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 09:04:29.009526 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 14 09:04:29.009526 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 176/340] 10.222.0.26 () {56 vars in 1114 bytes} [Thu Jul 14 09:04:28 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 09:04:29.023471 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c8984e-6814-4727-95cb-ee4788ea352b None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:29.039640 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464b7f3a-a344-4c98-8b06-92faa5a230a9 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:29.127754 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ffe5db-f92f-4b1f-8e75-581585e3598b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:29.128831 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ffe5db-f92f-4b1f-8e75-581585e3598b None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:29.129290 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ffe5db-f92f-4b1f-8e75-581585e3598b None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:29.129779 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ffe5db-f92f-4b1f-8e75-581585e3598b None admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:29.130768 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ffe5db-f92f-4b1f-8e75-581585e3598b None admin] RBAC: Adding query filter params (name=member) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:04:29.131288 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ffe5db-f92f-4b1f-8e75-581585e3598b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:29.132877 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ffe5db-f92f-4b1f-8e75-581585e3598b None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:29.151957 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 165/341] 10.222.0.26 () {56 vars in 1101 bytes} [Thu Jul 14 09:04:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:04:29.157760 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ffe5db-f92f-4b1f-8e75-581585e3598b None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:29.204537 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c8984e-6814-4727-95cb-ee4788ea352b None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:29.279579 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ec0a19-005c-49e2-b93f-13bf27a66650 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:29.280255 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ec0a19-005c-49e2-b93f-13bf27a66650 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:29.280548 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ec0a19-005c-49e2-b93f-13bf27a66650 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:29.280825 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ec0a19-005c-49e2-b93f-13bf27a66650 None admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:29.281594 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ec0a19-005c-49e2-b93f-13bf27a66650 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:04:29.281903 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ec0a19-005c-49e2-b93f-13bf27a66650 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:29.283316 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ec0a19-005c-49e2-b93f-13bf27a66650 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:29.300608 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 177/342] 10.222.0.26 () {56 vars in 1111 bytes} [Thu Jul 14 09:04:29 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:04:29.305261 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ec0a19-005c-49e2-b93f-13bf27a66650 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:29.366546 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ffe5db-f92f-4b1f-8e75-581585e3598b None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:29.409762 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d9d533-135f-4273-928f-1cfd85dc9298 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:29.411096 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d9d533-135f-4273-928f-1cfd85dc9298 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:29.411552 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d9d533-135f-4273-928f-1cfd85dc9298 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:29.411972 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d9d533-135f-4273-928f-1cfd85dc9298 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:29.415378 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:29.444981 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:29.494904 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d9d533-135f-4273-928f-1cfd85dc9298 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:29.497586 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d9d533-135f-4273-928f-1cfd85dc9298 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:29.505175 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ec0a19-005c-49e2-b93f-13bf27a66650 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:29.546757 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a083aa-bed6-4f3f-b5b5-bae7be7f43f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:29.547716 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a083aa-bed6-4f3f-b5b5-bae7be7f43f5 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:29.548452 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a083aa-bed6-4f3f-b5b5-bae7be7f43f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:29.548927 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a083aa-bed6-4f3f-b5b5-bae7be7f43f5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:29.626107 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 166/343] 10.222.0.26 () {58 vars in 1282 bytes} [Thu Jul 14 09:04:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 469 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 14 09:04:29.629885 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703549ae-e778-4e57-8e78-23d83131cd70 None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:29.630144 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703549ae-e778-4e57-8e78-23d83131cd70 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:29.630518 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703549ae-e778-4e57-8e78-23d83131cd70 None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:29.631853 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 167/344] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:04:29 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:04:29.633340 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f922065f-1234-47b4-8999-b4e627a1e998 None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:29.633839 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f922065f-1234-47b4-8999-b4e627a1e998 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:29.634332 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f922065f-1234-47b4-8999-b4e627a1e998 None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:29.635432 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a083aa-bed6-4f3f-b5b5-bae7be7f43f5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:29.636424 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 168/345] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:04:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:04:29.637218 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a083aa-bed6-4f3f-b5b5-bae7be7f43f5 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:29.637637 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f922065f-1234-47b4-8999-b4e627a1e998 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:29.680836 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:29.710862 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:29.761761 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 178/346] 10.222.0.26 () {58 vars in 1282 bytes} [Thu Jul 14 09:04:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 455 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 09:04:29.763369 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afae058c-3b9c-4226-b2e9-1048ccccd69e None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:29.763801 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afae058c-3b9c-4226-b2e9-1048ccccd69e None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:29.764220 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afae058c-3b9c-4226-b2e9-1048ccccd69e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:29.767064 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f922065f-1234-47b4-8999-b4e627a1e998 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:29.781852 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-afae058c-3b9c-4226-b2e9-1048ccccd69e None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:04:29.783354 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-afae058c-3b9c-4226-b2e9-1048ccccd69e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:04:29.809136 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9f92a8-2207-4529-aab8-59b93d2c5a8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:29.810360 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9f92a8-2207-4529-aab8-59b93d2c5a8e None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:29.810881 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9f92a8-2207-4529-aab8-59b93d2c5a8e None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:29.811321 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9f92a8-2207-4529-aab8-59b93d2c5a8e None admin] PATH_INFO: `/v3/projects/4a4ff416931141ce83908e05cc434590` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:29.813856 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9f92a8-2207-4529-aab8-59b93d2c5a8e None admin] RBAC: Authorizing `identity:get_project(project_id=4a4ff416931141ce83908e05cc434590)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:29.815503 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9f92a8-2207-4529-aab8-59b93d2c5a8e None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:29.817960 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 169/347] 10.222.0.26 () {56 vars in 1186 bytes} [Thu Jul 14 09:04:29 2022] GET /identity/v3/projects/4a4ff416931141ce83908e05cc434590 => generated 298 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:04:29.819381 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682eb229-c65d-4e65-80e4-ab8798fc8a36 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:29.819901 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682eb229-c65d-4e65-80e4-ab8798fc8a36 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:29.820411 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682eb229-c65d-4e65-80e4-ab8798fc8a36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:29.842662 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-682eb229-c65d-4e65-80e4-ab8798fc8a36 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:04:29.844151 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-682eb229-c65d-4e65-80e4-ab8798fc8a36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:04:29.856116 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 179/348] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:04:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 09:04:29.857046 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afae058c-3b9c-4226-b2e9-1048ccccd69e None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:29.919074 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 170/349] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:04:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 09:04:29.920183 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682eb229-c65d-4e65-80e4-ab8798fc8a36 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:29.947075 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:29.949073 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:29.964688 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afae058c-3b9c-4226-b2e9-1048ccccd69e None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:30.002896 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9beef0-c23b-4661-b781-0e3d80e1de21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:30.004216 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9beef0-c23b-4661-b781-0e3d80e1de21 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:30.004710 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9beef0-c23b-4661-b781-0e3d80e1de21 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:30.005196 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9beef0-c23b-4661-b781-0e3d80e1de21 None admin] PATH_INFO: `/v3/projects/4a4ff416931141ce83908e05cc434590` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:30.006709 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9beef0-c23b-4661-b781-0e3d80e1de21 None admin] RBAC: Authorizing `identity:get_project(project_id=4a4ff416931141ce83908e05cc434590)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:30.008431 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9beef0-c23b-4661-b781-0e3d80e1de21 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:30.010854 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 180/350] 10.222.0.26 () {56 vars in 1186 bytes} [Thu Jul 14 09:04:29 2022] GET /identity/v3/projects/4a4ff416931141ce83908e05cc434590 => generated 298 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:04:30.012000 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9beef0-c23b-4661-b781-0e3d80e1de21 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:30.030069 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682eb229-c65d-4e65-80e4-ab8798fc8a36 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:30.068591 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d681b5-5b15-43f7-861e-e05b3858d260 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:30.069700 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d681b5-5b15-43f7-861e-e05b3858d260 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:30.070321 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d681b5-5b15-43f7-861e-e05b3858d260 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:30.073421 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d681b5-5b15-43f7-861e-e05b3858d260 None admin] PATH_INFO: `/v3/groups/f2aa80870e8543bab7b8913e4d9de132` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:30.080295 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d681b5-5b15-43f7-861e-e05b3858d260 None admin] RBAC: Authorizing `identity:get_group(group_id=f2aa80870e8543bab7b8913e4d9de132)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:30.082215 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d681b5-5b15-43f7-861e-e05b3858d260 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:30.084272 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 171/351] 10.222.0.26 () {56 vars in 1180 bytes} [Thu Jul 14 09:04:29 2022] GET /identity/v3/groups/f2aa80870e8543bab7b8913e4d9de132 => generated 226 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:04:30.085309 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d681b5-5b15-43f7-861e-e05b3858d260 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:30.128128 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9beef0-c23b-4661-b781-0e3d80e1de21 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:30.165244 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7874a98a-08e8-4803-8a78-de0a61486669 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:30.166108 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7874a98a-08e8-4803-8a78-de0a61486669 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:30.166391 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7874a98a-08e8-4803-8a78-de0a61486669 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:30.166669 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7874a98a-08e8-4803-8a78-de0a61486669 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:30.179171 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7874a98a-08e8-4803-8a78-de0a61486669 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 09:04:30.182685 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:30.189168 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-7874a98a-08e8-4803-8a78-de0a61486669 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 09:04:30.189168 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:04:30.189168 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:04:30.189168 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:04:30.189168 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:04:30.189168 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:04:30.189168 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:04:30.189168 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:04:30.189168 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:04:30.189168 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:04:30.189168 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:04:30.189168 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:04:30.189168 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 09:04:30.189168 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 09:04:30.189168 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 09:04:30.189168 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 09:04:30.190300 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 09:04:30.190300 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 09:04:30.190300 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 09:04:30.190300 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 09:04:30.190300 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:04:30.190300 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:04:30.190300 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:04:30.190300 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:04:30.190300 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:04:30.190300 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:04:30.190300 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:04:30.190300 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 14 09:04:30.190300 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:04:30.190300 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:04:30.190300 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:04:30.190300 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:04:30.191357 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:04:30.191357 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:04:30.191357 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:04:30.191357 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:04:30.191357 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 09:04:30.191357 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 09:04:30.191357 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 09:04:30.191357 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 09:04:30.191357 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 09:04:30.191357 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 09:04:30.191357 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 09:04:30.191357 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 14 09:04:30.191357 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 181/352] 10.222.0.26 () {56 vars in 1099 bytes} [Thu Jul 14 09:04:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:04:30.191357 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7874a98a-08e8-4803-8a78-de0a61486669 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:30.204361 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d681b5-5b15-43f7-861e-e05b3858d260 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:30.214761 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:30.243086 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1561c4f-019f-47de-b24e-55c029bcb187 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:30.244245 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1561c4f-019f-47de-b24e-55c029bcb187 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:30.244676 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1561c4f-019f-47de-b24e-55c029bcb187 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:30.245097 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1561c4f-019f-47de-b24e-55c029bcb187 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:30.257567 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1561c4f-019f-47de-b24e-55c029bcb187 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 09:04:30.266276 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-b1561c4f-019f-47de-b24e-55c029bcb187 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 09:04:30.266276 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:04:30.266276 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:04:30.266276 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:04:30.266276 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:04:30.266276 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:04:30.266276 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:04:30.266276 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:04:30.266276 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:04:30.266276 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:04:30.266276 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:04:30.266276 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:04:30.266276 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 09:04:30.266276 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 09:04:30.266276 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 09:04:30.266276 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 09:04:30.267429 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 09:04:30.267429 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 09:04:30.267429 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 09:04:30.267429 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 09:04:30.267429 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:04:30.267429 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:04:30.267429 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:04:30.267429 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:04:30.267429 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:04:30.267429 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:04:30.267429 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:04:30.267429 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 14 09:04:30.267429 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:04:30.267429 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:04:30.267429 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:04:30.267429 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:04:30.268348 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:04:30.268348 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:04:30.268348 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:04:30.268348 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:04:30.268348 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 09:04:30.268348 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 09:04:30.268348 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 09:04:30.268348 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 09:04:30.268348 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 09:04:30.268348 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 09:04:30.268348 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 09:04:30.268348 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 14 09:04:30.270459 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 172/353] 10.222.0.26 () {56 vars in 1099 bytes} [Thu Jul 14 09:04:30 2022] GET /identity/v3/roles/reader => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:04:30.271453 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1561c4f-019f-47de-b24e-55c029bcb187 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:30.324379 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7874a98a-08e8-4803-8a78-de0a61486669 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:30.361410 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47303f3e-aa7e-42c5-94f9-380ed3d31f9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:30.362372 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47303f3e-aa7e-42c5-94f9-380ed3d31f9d None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:30.362813 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47303f3e-aa7e-42c5-94f9-380ed3d31f9d None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:30.363155 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47303f3e-aa7e-42c5-94f9-380ed3d31f9d None admin] PATH_INFO: `/v3/groups/f2aa80870e8543bab7b8913e4d9de132` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:30.364572 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47303f3e-aa7e-42c5-94f9-380ed3d31f9d None admin] RBAC: Authorizing `identity:get_group(group_id=f2aa80870e8543bab7b8913e4d9de132)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:30.366095 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47303f3e-aa7e-42c5-94f9-380ed3d31f9d None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:30.369514 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 182/354] 10.222.0.26 () {56 vars in 1180 bytes} [Thu Jul 14 09:04:30 2022] GET /identity/v3/groups/f2aa80870e8543bab7b8913e4d9de132 => generated 226 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:04:30.370453 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47303f3e-aa7e-42c5-94f9-380ed3d31f9d None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:30.377418 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1561c4f-019f-47de-b24e-55c029bcb187 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:30.411956 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7b4de2-29a6-418b-8929-327ab692899f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:30.413356 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7b4de2-29a6-418b-8929-327ab692899f None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:30.413842 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7b4de2-29a6-418b-8929-327ab692899f None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:30.414292 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7b4de2-29a6-418b-8929-327ab692899f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:30.415338 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7b4de2-29a6-418b-8929-327ab692899f None admin] RBAC: Adding query filter params (group.id=f2aa80870e8543bab7b8913e4d9de132, role.id=d1691322ce6a4edab7ce722edf0f2988, scope.project.id=4a4ff416931141ce83908e05cc434590) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:04:30.415829 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7b4de2-29a6-418b-8929-327ab692899f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:30.417473 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7b4de2-29a6-418b-8929-327ab692899f None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:30.424231 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 173/355] 10.222.0.26 () {56 vars in 1376 bytes} [Thu Jul 14 09:04:30 2022] GET /identity/v3/role_assignments?group.id=f2aa80870e8543bab7b8913e4d9de132&scope.project.id=4a4ff416931141ce83908e05cc434590&role.id=d1691322ce6a4edab7ce722edf0f2988 => generated 260 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:04:30.425370 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7b4de2-29a6-418b-8929-327ab692899f None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:30.442661 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:30.467525 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:30.489592 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47303f3e-aa7e-42c5-94f9-380ed3d31f9d None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:30.535917 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7b4de2-29a6-418b-8929-327ab692899f None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:30.566003 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c361cc-1fe3-413b-bbd0-c2e38fefd1d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:30.567014 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c361cc-1fe3-413b-bbd0-c2e38fefd1d4 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:30.567533 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c361cc-1fe3-413b-bbd0-c2e38fefd1d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:30.567972 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c361cc-1fe3-413b-bbd0-c2e38fefd1d4 None admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:30.568891 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c361cc-1fe3-413b-bbd0-c2e38fefd1d4 None admin] RBAC: Adding query filter params (name=member) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:04:30.569355 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c361cc-1fe3-413b-bbd0-c2e38fefd1d4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:30.571097 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c361cc-1fe3-413b-bbd0-c2e38fefd1d4 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:30.584408 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 183/356] 10.222.0.26 () {56 vars in 1101 bytes} [Thu Jul 14 09:04:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:04:30.586128 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c361cc-1fe3-413b-bbd0-c2e38fefd1d4 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:30.587042 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef7dd6f-aee0-454c-b6a7-0290223007d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:30.588370 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef7dd6f-aee0-454c-b6a7-0290223007d5 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:30.591075 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef7dd6f-aee0-454c-b6a7-0290223007d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:30.591577 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef7dd6f-aee0-454c-b6a7-0290223007d5 None admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:30.592515 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef7dd6f-aee0-454c-b6a7-0290223007d5 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:04:30.592991 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef7dd6f-aee0-454c-b6a7-0290223007d5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:30.594816 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef7dd6f-aee0-454c-b6a7-0290223007d5 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:30.619650 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 174/357] 10.222.0.26 () {56 vars in 1101 bytes} [Thu Jul 14 09:04:30 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:04:30.621883 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef7dd6f-aee0-454c-b6a7-0290223007d5 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:30.725106 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:30.763965 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef7dd6f-aee0-454c-b6a7-0290223007d5 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:30.770891 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c361cc-1fe3-413b-bbd0-c2e38fefd1d4 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:30.836104 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7d2875-124d-436d-89e3-d0627d07e8d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:30.837783 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7d2875-124d-436d-89e3-d0627d07e8d5 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:30.838762 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7d2875-124d-436d-89e3-d0627d07e8d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:30.839775 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7d2875-124d-436d-89e3-d0627d07e8d5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:30.842790 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb6ded6-fa01-47af-9c8a-85b797ca86c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:30.845400 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb6ded6-fa01-47af-9c8a-85b797ca86c9 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:30.845893 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb6ded6-fa01-47af-9c8a-85b797ca86c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:30.846433 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb6ded6-fa01-47af-9c8a-85b797ca86c9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:30.847746 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb6ded6-fa01-47af-9c8a-85b797ca86c9 None admin] RBAC: Adding query filter params (group.id=f2aa80870e8543bab7b8913e4d9de132, role.id=c4bab4a02e5c4f5f97a024e844a0c072, scope.project.id=4a4ff416931141ce83908e05cc434590) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:04:30.849187 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb6ded6-fa01-47af-9c8a-85b797ca86c9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:30.850863 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb6ded6-fa01-47af-9c8a-85b797ca86c9 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:30.857471 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 184/358] 10.222.0.26 () {56 vars in 1376 bytes} [Thu Jul 14 09:04:30 2022] GET /identity/v3/role_assignments?group.id=f2aa80870e8543bab7b8913e4d9de132&scope.project.id=4a4ff416931141ce83908e05cc434590&role.id=c4bab4a02e5c4f5f97a024e844a0c072 => generated 260 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:04:30.858683 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb6ded6-fa01-47af-9c8a-85b797ca86c9 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:30.883809 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:30.957612 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7d2875-124d-436d-89e3-d0627d07e8d5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:30.959593 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7d2875-124d-436d-89e3-d0627d07e8d5 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:31.015378 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb6ded6-fa01-47af-9c8a-85b797ca86c9 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:31.048341 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:31.058282 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b2ab21-ee20-401f-8d03-38ebbe8acbc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:31.059684 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b2ab21-ee20-401f-8d03-38ebbe8acbc2 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:31.060462 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b2ab21-ee20-401f-8d03-38ebbe8acbc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:31.060954 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b2ab21-ee20-401f-8d03-38ebbe8acbc2 None admin] PATH_INFO: `/v3/users/30bb7e4926d944aeb839cc90e9c8ab7f` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:31.062398 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b2ab21-ee20-401f-8d03-38ebbe8acbc2 None admin] RBAC: Authorizing `identity:get_user(user_id=30bb7e4926d944aeb839cc90e9c8ab7f)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:31.064567 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b2ab21-ee20-401f-8d03-38ebbe8acbc2 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:31.067347 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 185/359] 10.222.0.26 () {56 vars in 1177 bytes} [Thu Jul 14 09:04:30 2022] GET /identity/v3/users/30bb7e4926d944aeb839cc90e9c8ab7f => generated 293 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:04:31.068855 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28ef7bc-54e9-44ef-a528-33ab6e4eb598 None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:31.069336 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28ef7bc-54e9-44ef-a528-33ab6e4eb598 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:31.069800 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28ef7bc-54e9-44ef-a528-33ab6e4eb598 None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:31.071522 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 186/360] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:04:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:04:31.072981 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd474e8a-3e39-46b2-a608-94a81eaf101c None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:31.073483 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd474e8a-3e39-46b2-a608-94a81eaf101c None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:31.073946 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd474e8a-3e39-46b2-a608-94a81eaf101c None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:31.077588 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 187/361] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:04:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:04:31.077588 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd474e8a-3e39-46b2-a608-94a81eaf101c None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:31.085258 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 175/362] 10.222.0.26 () {58 vars in 1282 bytes} [Thu Jul 14 09:04:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 464 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 09:04:31.086626 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d86795-2958-47bb-887e-36cccba1e48a None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:31.086860 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d86795-2958-47bb-887e-36cccba1e48a None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:31.087134 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d86795-2958-47bb-887e-36cccba1e48a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:31.109475 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-f0d86795-2958-47bb-887e-36cccba1e48a None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:04:31.110660 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-f0d86795-2958-47bb-887e-36cccba1e48a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:04:31.187650 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:31.198888 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 176/363] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:04:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 09:04:31.200633 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd39a21d-749f-49fe-b7ce-ad284b6264e0 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:31.201057 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd39a21d-749f-49fe-b7ce-ad284b6264e0 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:31.201468 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd39a21d-749f-49fe-b7ce-ad284b6264e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:31.204900 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd474e8a-3e39-46b2-a608-94a81eaf101c None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:31.230673 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-fd39a21d-749f-49fe-b7ce-ad284b6264e0 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:04:31.232294 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-fd39a21d-749f-49fe-b7ce-ad284b6264e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:04:31.243861 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3539ab7-ae45-41c1-a1ab-1141c09d541f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:31.244856 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3539ab7-ae45-41c1-a1ab-1141c09d541f None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:31.245368 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3539ab7-ae45-41c1-a1ab-1141c09d541f None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:31.246286 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3539ab7-ae45-41c1-a1ab-1141c09d541f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:31.247356 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3539ab7-ae45-41c1-a1ab-1141c09d541f None admin] RBAC: Adding query filter params (role.id=9c412baac5914ce6a40ad7d3f0f0a0d6, user.id=30bb7e4926d944aeb839cc90e9c8ab7f, scope.system=all) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:04:31.247866 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3539ab7-ae45-41c1-a1ab-1141c09d541f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:31.249515 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3539ab7-ae45-41c1-a1ab-1141c09d541f None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:31.265811 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 188/364] 10.222.0.26 () {56 vars in 1308 bytes} [Thu Jul 14 09:04:31 2022] GET /identity/v3/role_assignments?user.id=30bb7e4926d944aeb839cc90e9c8ab7f&scope.system=all&role.id=9c412baac5914ce6a40ad7d3f0f0a0d6 => generated 510 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:04:31.267819 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3539ab7-ae45-41c1-a1ab-1141c09d541f None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:31.327104 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 177/365] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:04:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 09:04:31.327104 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd39a21d-749f-49fe-b7ce-ad284b6264e0 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:31.381680 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:31.446066 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3539ab7-ae45-41c1-a1ab-1141c09d541f None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:31.459813 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd39a21d-749f-49fe-b7ce-ad284b6264e0 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:31.490721 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb82091-f2fd-4784-ba66-8a46ddf79d04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:31.491848 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb82091-f2fd-4784-ba66-8a46ddf79d04 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:31.492303 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb82091-f2fd-4784-ba66-8a46ddf79d04 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:31.492817 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb82091-f2fd-4784-ba66-8a46ddf79d04 None admin] PATH_INFO: `/v3/projects/a596189576264bfc9638ca33f184759c` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:31.494369 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb82091-f2fd-4784-ba66-8a46ddf79d04 None admin] RBAC: Authorizing `identity:get_project(project_id=a596189576264bfc9638ca33f184759c)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:31.498589 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb82091-f2fd-4784-ba66-8a46ddf79d04 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:31.499592 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7751162e-c927-460a-858d-05da754ccf47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:31.500551 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7751162e-c927-460a-858d-05da754ccf47 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:31.501043 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7751162e-c927-460a-858d-05da754ccf47 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:31.501398 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 189/366] 10.222.0.26 () {56 vars in 1186 bytes} [Thu Jul 14 09:04:31 2022] GET /identity/v3/projects/a596189576264bfc9638ca33f184759c => generated 302 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:04:31.501745 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7751162e-c927-460a-858d-05da754ccf47 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:31.504723 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb82091-f2fd-4784-ba66-8a46ddf79d04 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:31.506535 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:31.521571 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7751162e-c927-460a-858d-05da754ccf47 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 09:04:31.530878 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-7751162e-c927-460a-858d-05da754ccf47 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 09:04:31.530878 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:04:31.530878 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:04:31.530878 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:04:31.530878 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:04:31.530878 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:04:31.530878 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:04:31.530878 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:04:31.530878 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:04:31.530878 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:04:31.530878 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:04:31.530878 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:04:31.530878 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 09:04:31.530878 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 09:04:31.530878 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 09:04:31.530878 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 09:04:31.531830 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 09:04:31.531830 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 09:04:31.531830 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 09:04:31.531830 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 09:04:31.531830 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:04:31.531830 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:04:31.531830 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:04:31.531830 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:04:31.531830 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:04:31.531830 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:04:31.531830 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:04:31.531830 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 14 09:04:31.531830 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:04:31.531830 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:04:31.531830 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:04:31.531830 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:04:31.532645 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:04:31.532645 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:04:31.532645 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:04:31.532645 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:04:31.532645 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 09:04:31.532645 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 09:04:31.532645 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 09:04:31.532645 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 09:04:31.532645 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 09:04:31.532645 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 09:04:31.532645 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 09:04:31.532645 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 14 09:04:31.538549 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 178/367] 10.222.0.26 () {56 vars in 1096 bytes} [Thu Jul 14 09:04:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:04:31.539735 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7751162e-c927-460a-858d-05da754ccf47 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:31.633977 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb82091-f2fd-4784-ba66-8a46ddf79d04 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:31.647787 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:31.656287 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7751162e-c927-460a-858d-05da754ccf47 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:31.684286 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3052f192-2fbd-4c1b-b292-fa395f5132a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:31.685507 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3052f192-2fbd-4c1b-b292-fa395f5132a9 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:31.685507 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3052f192-2fbd-4c1b-b292-fa395f5132a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:31.685759 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3052f192-2fbd-4c1b-b292-fa395f5132a9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:31.697436 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88df48f5-de02-4a81-93d6-cf3b6c5d569a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:31.698753 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88df48f5-de02-4a81-93d6-cf3b6c5d569a None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:31.699283 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88df48f5-de02-4a81-93d6-cf3b6c5d569a None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:31.699748 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88df48f5-de02-4a81-93d6-cf3b6c5d569a None admin] PATH_INFO: `/v3/groups/f2aa80870e8543bab7b8913e4d9de132` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:31.701221 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88df48f5-de02-4a81-93d6-cf3b6c5d569a None admin] RBAC: Authorizing `identity:get_group(group_id=f2aa80870e8543bab7b8913e4d9de132)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:31.701325 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3052f192-2fbd-4c1b-b292-fa395f5132a9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 09:04:31.703018 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88df48f5-de02-4a81-93d6-cf3b6c5d569a None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:31.705195 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 179/368] 10.222.0.26 () {56 vars in 1180 bytes} [Thu Jul 14 09:04:31 2022] GET /identity/v3/groups/f2aa80870e8543bab7b8913e4d9de132 => generated 226 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:04:31.706277 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88df48f5-de02-4a81-93d6-cf3b6c5d569a None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:31.710817 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-3052f192-2fbd-4c1b-b292-fa395f5132a9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 09:04:31.710817 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:04:31.710817 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:04:31.710817 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:04:31.710817 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:04:31.710817 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:04:31.710817 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:04:31.710817 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:04:31.710817 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:04:31.710817 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:04:31.710817 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:04:31.710817 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:04:31.710817 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 09:04:31.710817 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 09:04:31.710817 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 09:04:31.710817 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 09:04:31.711737 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 09:04:31.711737 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 09:04:31.711737 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 09:04:31.711737 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 09:04:31.711737 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:04:31.711737 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:04:31.711737 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:04:31.711737 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:04:31.711737 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:04:31.711737 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:04:31.711737 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:04:31.711737 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 14 09:04:31.711737 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:04:31.711737 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:04:31.711737 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:04:31.711737 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:04:31.712531 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:04:31.712531 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:04:31.712531 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:04:31.712531 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:04:31.712531 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 09:04:31.712531 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 09:04:31.712531 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 09:04:31.712531 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 09:04:31.712531 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 09:04:31.712531 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 09:04:31.712531 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 09:04:31.712531 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 14 09:04:31.712531 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 190/369] 10.222.0.26 () {56 vars in 1114 bytes} [Thu Jul 14 09:04:31 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:04:31.720011 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3052f192-2fbd-4c1b-b292-fa395f5132a9 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:31.800290 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:31.843891 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3052f192-2fbd-4c1b-b292-fa395f5132a9 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:31.851071 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88df48f5-de02-4a81-93d6-cf3b6c5d569a None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:31.885026 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d439d3d1-c645-4bee-990b-ab6d4e58a4a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:31.885772 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d439d3d1-c645-4bee-990b-ab6d4e58a4a0 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:31.886149 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d439d3d1-c645-4bee-990b-ab6d4e58a4a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:31.886518 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d439d3d1-c645-4bee-990b-ab6d4e58a4a0 None admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:31.887437 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d439d3d1-c645-4bee-990b-ab6d4e58a4a0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:04:31.887835 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d439d3d1-c645-4bee-990b-ab6d4e58a4a0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:31.889339 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d439d3d1-c645-4bee-990b-ab6d4e58a4a0 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:31.897240 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c3c326-b6aa-4c5c-8ae5-bcdea40de6dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:31.898385 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c3c326-b6aa-4c5c-8ae5-bcdea40de6dd None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:31.898805 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c3c326-b6aa-4c5c-8ae5-bcdea40de6dd None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:31.899238 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c3c326-b6aa-4c5c-8ae5-bcdea40de6dd None admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:31.900146 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c3c326-b6aa-4c5c-8ae5-bcdea40de6dd None admin] RBAC: Adding query filter params (name=admin) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:04:31.900366 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 191/370] 10.222.0.26 () {56 vars in 1111 bytes} [Thu Jul 14 09:04:31 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:04:31.900731 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c3c326-b6aa-4c5c-8ae5-bcdea40de6dd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:31.901590 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d439d3d1-c645-4bee-990b-ab6d4e58a4a0 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:31.902407 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c3c326-b6aa-4c5c-8ae5-bcdea40de6dd None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:31.915082 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 180/371] 10.222.0.26 () {56 vars in 1099 bytes} [Thu Jul 14 09:04:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:04:31.916526 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c3c326-b6aa-4c5c-8ae5-bcdea40de6dd None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:31.949713 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:32.027409 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c3c326-b6aa-4c5c-8ae5-bcdea40de6dd None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:32.031071 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d439d3d1-c645-4bee-990b-ab6d4e58a4a0 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:32.062506 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:32.071641 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0643d786-2b7c-40a0-b32a-9cf02357436c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:32.071641 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0643d786-2b7c-40a0-b32a-9cf02357436c None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:32.071641 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0643d786-2b7c-40a0-b32a-9cf02357436c None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:32.071949 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0643d786-2b7c-40a0-b32a-9cf02357436c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:32.075973 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3652ad-b6f5-4678-aa8c-6a0c1c94dc3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:32.077621 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3652ad-b6f5-4678-aa8c-6a0c1c94dc3c None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:32.078051 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3652ad-b6f5-4678-aa8c-6a0c1c94dc3c None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:32.078463 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3652ad-b6f5-4678-aa8c-6a0c1c94dc3c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:32.079474 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3652ad-b6f5-4678-aa8c-6a0c1c94dc3c None admin] RBAC: Adding query filter params (group.id=f2aa80870e8543bab7b8913e4d9de132, role.id=d1691322ce6a4edab7ce722edf0f2988, scope.project.id=a596189576264bfc9638ca33f184759c) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:04:32.079923 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3652ad-b6f5-4678-aa8c-6a0c1c94dc3c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:32.081550 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3652ad-b6f5-4678-aa8c-6a0c1c94dc3c None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:32.089153 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 192/372] 10.222.0.26 () {56 vars in 1376 bytes} [Thu Jul 14 09:04:31 2022] GET /identity/v3/role_assignments?group.id=f2aa80870e8543bab7b8913e4d9de132&scope.project.id=a596189576264bfc9638ca33f184759c&role.id=d1691322ce6a4edab7ce722edf0f2988 => generated 260 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:04:32.090165 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3652ad-b6f5-4678-aa8c-6a0c1c94dc3c None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:32.215694 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0643d786-2b7c-40a0-b32a-9cf02357436c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:32.217113 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0643d786-2b7c-40a0-b32a-9cf02357436c None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:32.253852 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:32.269222 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3652ad-b6f5-4678-aa8c-6a0c1c94dc3c None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:32.336769 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9f7f40-a5ec-4619-8c7d-de9147ccef9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:32.337440 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9f7f40-a5ec-4619-8c7d-de9147ccef9f None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:32.337716 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9f7f40-a5ec-4619-8c7d-de9147ccef9f None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:32.337987 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9f7f40-a5ec-4619-8c7d-de9147ccef9f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:32.387829 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 181/373] 10.222.0.26 () {58 vars in 1282 bytes} [Thu Jul 14 09:04:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 471 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 09:04:32.393309 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0643d786-2b7c-40a0-b32a-9cf02357436c None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:32.446894 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:32.472736 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9f7f40-a5ec-4619-8c7d-de9147ccef9f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:32.474167 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9f7f40-a5ec-4619-8c7d-de9147ccef9f None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:32.513380 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0643d786-2b7c-40a0-b32a-9cf02357436c None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:32.546114 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:32.559977 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c011173f-abd2-44e4-81b0-dda372aecf39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:32.561179 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c011173f-abd2-44e4-81b0-dda372aecf39 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:32.561673 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c011173f-abd2-44e4-81b0-dda372aecf39 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:32.562191 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c011173f-abd2-44e4-81b0-dda372aecf39 None admin] PATH_INFO: `/v3/projects/a596189576264bfc9638ca33f184759c` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:32.564125 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c011173f-abd2-44e4-81b0-dda372aecf39 None admin] RBAC: Authorizing `identity:get_project(project_id=a596189576264bfc9638ca33f184759c)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:32.565806 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c011173f-abd2-44e4-81b0-dda372aecf39 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:32.568175 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 182/374] 10.222.0.26 () {56 vars in 1186 bytes} [Thu Jul 14 09:04:32 2022] GET /identity/v3/projects/a596189576264bfc9638ca33f184759c => generated 302 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:04:32.574194 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c011173f-abd2-44e4-81b0-dda372aecf39 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:32.688579 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c011173f-abd2-44e4-81b0-dda372aecf39 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:32.713412 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 193/375] 10.222.0.26 () {58 vars in 1282 bytes} [Thu Jul 14 09:04:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 624 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 09:04:32.719264 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9f7f40-a5ec-4619-8c7d-de9147ccef9f None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:32.742296 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b38f76-780e-455b-90fd-faf142b3494d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:32.743092 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b38f76-780e-455b-90fd-faf142b3494d None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:32.743414 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b38f76-780e-455b-90fd-faf142b3494d None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:32.743687 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b38f76-780e-455b-90fd-faf142b3494d None admin] PATH_INFO: `/v3/groups/f2aa80870e8543bab7b8913e4d9de132` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:32.744864 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b38f76-780e-455b-90fd-faf142b3494d None admin] RBAC: Authorizing `identity:get_group(group_id=f2aa80870e8543bab7b8913e4d9de132)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:32.746325 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b38f76-780e-455b-90fd-faf142b3494d None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:32.748332 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 183/376] 10.222.0.26 () {56 vars in 1180 bytes} [Thu Jul 14 09:04:32 2022] GET /identity/v3/groups/f2aa80870e8543bab7b8913e4d9de132 => generated 226 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:04:32.754679 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b38f76-780e-455b-90fd-faf142b3494d None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:32.831791 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:32.888393 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:32.893861 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b38f76-780e-455b-90fd-faf142b3494d None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:32.908381 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9f7f40-a5ec-4619-8c7d-de9147ccef9f None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:32.962135 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1990c7-ab2e-4874-8f43-e7ca6f8a03cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:32.962798 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1990c7-ab2e-4874-8f43-e7ca6f8a03cb None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:32.963080 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1990c7-ab2e-4874-8f43-e7ca6f8a03cb None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:32.963383 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1990c7-ab2e-4874-8f43-e7ca6f8a03cb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:32.964224 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1990c7-ab2e-4874-8f43-e7ca6f8a03cb None admin] RBAC: Adding query filter params (group.id=f2aa80870e8543bab7b8913e4d9de132, role.id=c4bab4a02e5c4f5f97a024e844a0c072, scope.project.id=a596189576264bfc9638ca33f184759c) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:04:32.964532 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1990c7-ab2e-4874-8f43-e7ca6f8a03cb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:32.965976 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1990c7-ab2e-4874-8f43-e7ca6f8a03cb None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:32.973713 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8271db73-424e-4a86-8c7c-b2c2dbb1356b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:32.974591 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8271db73-424e-4a86-8c7c-b2c2dbb1356b None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:32.974872 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8271db73-424e-4a86-8c7c-b2c2dbb1356b None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:32.975143 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8271db73-424e-4a86-8c7c-b2c2dbb1356b None admin] PATH_INFO: `/v3/projects/2fd144ddf49f41d58d6674bed415082b` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:32.977502 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8271db73-424e-4a86-8c7c-b2c2dbb1356b None admin] RBAC: Authorizing `identity:get_project(project_id=2fd144ddf49f41d58d6674bed415082b)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:32.978946 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8271db73-424e-4a86-8c7c-b2c2dbb1356b None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:32.980730 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 194/377] 10.222.0.26 () {56 vars in 1186 bytes} [Thu Jul 14 09:04:32 2022] GET /identity/v3/projects/2fd144ddf49f41d58d6674bed415082b => generated 344 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 09:04:32.983826 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 184/378] 10.222.0.26 () {56 vars in 1376 bytes} [Thu Jul 14 09:04:32 2022] GET /identity/v3/role_assignments?group.id=f2aa80870e8543bab7b8913e4d9de132&scope.project.id=a596189576264bfc9638ca33f184759c&role.id=c4bab4a02e5c4f5f97a024e844a0c072 => generated 260 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 09:04:32.986093 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8271db73-424e-4a86-8c7c-b2c2dbb1356b None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:33.094103 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8271db73-424e-4a86-8c7c-b2c2dbb1356b None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:33.118510 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ec1fdb-1a7a-499e-8296-cfd9b4ce9a4e None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:33.118855 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ec1fdb-1a7a-499e-8296-cfd9b4ce9a4e None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:33.119060 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ec1fdb-1a7a-499e-8296-cfd9b4ce9a4e None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:33.120513 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 185/379] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:04:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:04:33.126244 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c738005-59ae-41c6-b48d-17a4bef4a930 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:33.126527 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c738005-59ae-41c6-b48d-17a4bef4a930 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:33.126801 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c738005-59ae-41c6-b48d-17a4bef4a930 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:33.152452 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae20909-4479-4f51-9bb5-1b6501353c4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:33.153342 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae20909-4479-4f51-9bb5-1b6501353c4c None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:33.153622 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae20909-4479-4f51-9bb5-1b6501353c4c None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:33.153892 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae20909-4479-4f51-9bb5-1b6501353c4c None admin] PATH_INFO: `/v3/groups/ae3d183691c04d43b40da20c4381678e` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:33.155539 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-8c738005-59ae-41c6-b48d-17a4bef4a930 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:04:33.156751 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-8c738005-59ae-41c6-b48d-17a4bef4a930 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:04:33.170722 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae20909-4479-4f51-9bb5-1b6501353c4c None admin] RBAC: Authorizing `identity:get_group(group_id=ae3d183691c04d43b40da20c4381678e)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:33.172522 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae20909-4479-4f51-9bb5-1b6501353c4c None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:33.176755 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 195/380] 10.222.0.26 () {56 vars in 1180 bytes} [Thu Jul 14 09:04:32 2022] GET /identity/v3/groups/ae3d183691c04d43b40da20c4381678e => generated 229 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:04:33.180154 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae20909-4479-4f51-9bb5-1b6501353c4c None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:33.233178 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:33.263105 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 186/381] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:04:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 09:04:33.287122 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c738005-59ae-41c6-b48d-17a4bef4a930 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:33.344202 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae20909-4479-4f51-9bb5-1b6501353c4c None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:33.402054 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:33.419689 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3699a565-ce91-4214-b5ef-315fa104eac4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:33.420369 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3699a565-ce91-4214-b5ef-315fa104eac4 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:33.420677 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3699a565-ce91-4214-b5ef-315fa104eac4 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:33.421461 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3699a565-ce91-4214-b5ef-315fa104eac4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:33.421883 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3699a565-ce91-4214-b5ef-315fa104eac4 None admin] RBAC: Adding query filter params (group.id=ae3d183691c04d43b40da20c4381678e, role.id=7b549d58d1724395b343609d895c80d0, scope.project.id=2fd144ddf49f41d58d6674bed415082b) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:04:33.422512 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3699a565-ce91-4214-b5ef-315fa104eac4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:33.424011 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3699a565-ce91-4214-b5ef-315fa104eac4 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:33.436531 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 196/382] 10.222.0.26 () {56 vars in 1376 bytes} [Thu Jul 14 09:04:33 2022] GET /identity/v3/role_assignments?group.id=ae3d183691c04d43b40da20c4381678e&scope.project.id=2fd144ddf49f41d58d6674bed415082b&role.id=7b549d58d1724395b343609d895c80d0 => generated 260 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:04:33.437936 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f1fcd2-6a07-49a7-9d15-9eefb514a9d7 None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:33.438254 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f1fcd2-6a07-49a7-9d15-9eefb514a9d7 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:33.443217 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f1fcd2-6a07-49a7-9d15-9eefb514a9d7 None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:33.443217 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 197/383] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:04:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:04:33.454226 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e057a356-f73e-454d-9a99-1a29e2d7eeee None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:33.454557 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e057a356-f73e-454d-9a99-1a29e2d7eeee None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:33.454782 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e057a356-f73e-454d-9a99-1a29e2d7eeee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:33.477484 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-e057a356-f73e-454d-9a99-1a29e2d7eeee None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:04:33.478709 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-e057a356-f73e-454d-9a99-1a29e2d7eeee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:04:33.479405 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c738005-59ae-41c6-b48d-17a4bef4a930 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:33.546481 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0a9cb8-ab2c-4452-8f20-69caad7c4799 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:33.548465 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0a9cb8-ab2c-4452-8f20-69caad7c4799 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:33.548984 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0a9cb8-ab2c-4452-8f20-69caad7c4799 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:33.549517 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0a9cb8-ab2c-4452-8f20-69caad7c4799 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:33.570745 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0a9cb8-ab2c-4452-8f20-69caad7c4799 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 09:04:33.572307 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 198/384] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:04:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 09:04:33.583651 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-fd0a9cb8-ab2c-4452-8f20-69caad7c4799 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 09:04:33.583651 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:04:33.583651 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:04:33.583651 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:04:33.583651 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:04:33.583651 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:04:33.583651 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:04:33.583651 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:04:33.583651 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:04:33.583651 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:04:33.583651 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:04:33.583651 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:04:33.583651 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 09:04:33.583651 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 09:04:33.583651 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 09:04:33.583651 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 09:04:33.584928 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 09:04:33.584928 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 09:04:33.584928 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 09:04:33.584928 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 09:04:33.584928 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:04:33.584928 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:04:33.584928 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:04:33.584928 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:04:33.584928 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:04:33.584928 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:04:33.584928 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:04:33.584928 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 14 09:04:33.584928 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:04:33.584928 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:04:33.584928 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:04:33.584928 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:04:33.585821 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:04:33.585821 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:04:33.585821 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:04:33.585821 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:04:33.585821 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 09:04:33.585821 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 09:04:33.585821 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 09:04:33.585821 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 09:04:33.585821 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 09:04:33.585821 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 09:04:33.585821 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 09:04:33.585821 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 14 09:04:33.588561 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 187/385] 10.222.0.26 () {56 vars in 1099 bytes} [Thu Jul 14 09:04:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 09:04:33.594988 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e057a356-f73e-454d-9a99-1a29e2d7eeee None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:33.599297 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0a9cb8-ab2c-4452-8f20-69caad7c4799 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:33.620793 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:33.783055 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e057a356-f73e-454d-9a99-1a29e2d7eeee None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:33.795349 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0a9cb8-ab2c-4452-8f20-69caad7c4799 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:33.831297 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:33.852568 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9417ada1-14f2-4327-a15b-8a9718211621 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:33.853239 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9417ada1-14f2-4327-a15b-8a9718211621 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:33.853516 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9417ada1-14f2-4327-a15b-8a9718211621 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:33.853793 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9417ada1-14f2-4327-a15b-8a9718211621 None admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:33.855967 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9417ada1-14f2-4327-a15b-8a9718211621 None admin] RBAC: Adding query filter params (name=member) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:04:33.856274 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9417ada1-14f2-4327-a15b-8a9718211621 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:33.857675 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9417ada1-14f2-4327-a15b-8a9718211621 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:33.863236 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4791404-1256-418b-88cb-dec1dcc703ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:33.864069 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4791404-1256-418b-88cb-dec1dcc703ee None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:33.864413 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4791404-1256-418b-88cb-dec1dcc703ee None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:33.864655 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4791404-1256-418b-88cb-dec1dcc703ee None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:33.878389 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 199/386] 10.222.0.26 () {56 vars in 1101 bytes} [Thu Jul 14 09:04:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:04:33.883515 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9417ada1-14f2-4327-a15b-8a9718211621 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:33.887852 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4791404-1256-418b-88cb-dec1dcc703ee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 09:04:33.902305 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-c4791404-1256-418b-88cb-dec1dcc703ee None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 09:04:33.902305 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:04:33.902305 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:04:33.902305 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:04:33.902305 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:04:33.902305 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:04:33.902305 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:04:33.902305 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:04:33.902305 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:04:33.902305 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:04:33.902305 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:04:33.902305 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:04:33.902305 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 09:04:33.902305 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 09:04:33.902305 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 09:04:33.902305 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 09:04:33.903298 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 09:04:33.903298 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 09:04:33.903298 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 09:04:33.903298 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 09:04:33.903298 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:04:33.903298 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:04:33.903298 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:04:33.903298 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:04:33.903298 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:04:33.903298 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:04:33.903298 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:04:33.903298 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 14 09:04:33.903298 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:04:33.903298 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:04:33.903298 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:04:33.903298 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:04:33.904102 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:04:33.904102 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:04:33.904102 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:04:33.904102 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:04:33.904102 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 09:04:33.904102 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 09:04:33.904102 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 09:04:33.904102 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 09:04:33.904102 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 09:04:33.904102 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 09:04:33.904102 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 09:04:33.904102 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 14 09:04:33.913190 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 188/387] 10.222.0.26 () {56 vars in 1114 bytes} [Thu Jul 14 09:04:33 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 09:04:33.917724 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4791404-1256-418b-88cb-dec1dcc703ee None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:33.989073 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:34.018634 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9417ada1-14f2-4327-a15b-8a9718211621 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:34.100780 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a26553f-efe7-40d0-b2e0-a410f9dee7c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:34.101662 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a26553f-efe7-40d0-b2e0-a410f9dee7c4 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:34.101937 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a26553f-efe7-40d0-b2e0-a410f9dee7c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:34.102208 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a26553f-efe7-40d0-b2e0-a410f9dee7c4 None admin] PATH_INFO: `/v3/groups/f2aa80870e8543bab7b8913e4d9de132` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:34.103486 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a26553f-efe7-40d0-b2e0-a410f9dee7c4 None admin] RBAC: Authorizing `identity:get_group(group_id=f2aa80870e8543bab7b8913e4d9de132)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:34.104871 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a26553f-efe7-40d0-b2e0-a410f9dee7c4 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:34.106511 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 200/388] 10.222.0.26 () {56 vars in 1180 bytes} [Thu Jul 14 09:04:33 2022] GET /identity/v3/groups/f2aa80870e8543bab7b8913e4d9de132 => generated 226 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:04:34.106826 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4791404-1256-418b-88cb-dec1dcc703ee None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:34.112132 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a26553f-efe7-40d0-b2e0-a410f9dee7c4 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:34.144202 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fdea75-e509-4ad0-88a5-966c80ff7710 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:34.145082 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fdea75-e509-4ad0-88a5-966c80ff7710 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:34.145374 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fdea75-e509-4ad0-88a5-966c80ff7710 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:34.145655 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fdea75-e509-4ad0-88a5-966c80ff7710 None admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:34.146426 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fdea75-e509-4ad0-88a5-966c80ff7710 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:04:34.146737 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fdea75-e509-4ad0-88a5-966c80ff7710 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:34.148222 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fdea75-e509-4ad0-88a5-966c80ff7710 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:34.156969 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 189/389] 10.222.0.26 () {56 vars in 1111 bytes} [Thu Jul 14 09:04:33 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:04:34.165524 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fdea75-e509-4ad0-88a5-966c80ff7710 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:34.203886 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:34.240879 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a26553f-efe7-40d0-b2e0-a410f9dee7c4 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:34.275978 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3b5e6c-8980-433c-b3b5-6d1ebce87654 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:34.276855 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3b5e6c-8980-433c-b3b5-6d1ebce87654 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:34.277127 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3b5e6c-8980-433c-b3b5-6d1ebce87654 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:34.277389 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3b5e6c-8980-433c-b3b5-6d1ebce87654 None admin] PATH_INFO: `/v3/projects/4a4ff416931141ce83908e05cc434590` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:34.278589 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3b5e6c-8980-433c-b3b5-6d1ebce87654 None admin] RBAC: Authorizing `identity:get_project(project_id=4a4ff416931141ce83908e05cc434590)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:34.280058 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3b5e6c-8980-433c-b3b5-6d1ebce87654 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:34.281767 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 201/390] 10.222.0.26 () {56 vars in 1186 bytes} [Thu Jul 14 09:04:34 2022] GET /identity/v3/projects/4a4ff416931141ce83908e05cc434590 => generated 298 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 09:04:34.287422 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3b5e6c-8980-433c-b3b5-6d1ebce87654 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:34.296138 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fdea75-e509-4ad0-88a5-966c80ff7710 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:34.306342 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:34.346678 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0306eb-514c-45ff-a07f-33926006de29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:34.347876 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0306eb-514c-45ff-a07f-33926006de29 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:34.348163 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0306eb-514c-45ff-a07f-33926006de29 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:34.348439 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0306eb-514c-45ff-a07f-33926006de29 None admin] PATH_INFO: `/v3/groups/f2aa80870e8543bab7b8913e4d9de132` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:34.349728 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0306eb-514c-45ff-a07f-33926006de29 None admin] RBAC: Authorizing `identity:get_group(group_id=f2aa80870e8543bab7b8913e4d9de132)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:34.351810 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0306eb-514c-45ff-a07f-33926006de29 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:34.354387 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 190/391] 10.222.0.26 () {56 vars in 1180 bytes} [Thu Jul 14 09:04:34 2022] GET /identity/v3/groups/f2aa80870e8543bab7b8913e4d9de132 => generated 226 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:04:34.360206 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0306eb-514c-45ff-a07f-33926006de29 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:34.468287 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3b5e6c-8980-433c-b3b5-6d1ebce87654 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:34.472094 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0306eb-514c-45ff-a07f-33926006de29 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:34.483544 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:34.509927 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5395b07-7bea-4ec2-97e0-28f16bc40248 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:34.510914 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5395b07-7bea-4ec2-97e0-28f16bc40248 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:34.511218 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5395b07-7bea-4ec2-97e0-28f16bc40248 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:34.511503 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5395b07-7bea-4ec2-97e0-28f16bc40248 None admin] PATH_INFO: `/v3/projects/4a4ff416931141ce83908e05cc434590` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:34.512878 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5395b07-7bea-4ec2-97e0-28f16bc40248 None admin] RBAC: Authorizing `identity:get_project(project_id=4a4ff416931141ce83908e05cc434590)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:34.515469 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5395b07-7bea-4ec2-97e0-28f16bc40248 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:34.517754 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 191/392] 10.222.0.26 () {56 vars in 1186 bytes} [Thu Jul 14 09:04:34 2022] GET /identity/v3/projects/4a4ff416931141ce83908e05cc434590 => generated 298 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:04:34.519949 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d677781c-87ed-40d8-9246-54081d7158e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:34.520856 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d677781c-87ed-40d8-9246-54081d7158e2 None admin] REQUEST_METHOD: `PUT` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:34.521132 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d677781c-87ed-40d8-9246-54081d7158e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:34.521402 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d677781c-87ed-40d8-9246-54081d7158e2 None admin] PATH_INFO: `/v3/projects/4a4ff416931141ce83908e05cc434590/groups/f2aa80870e8543bab7b8913e4d9de132/roles/d1691322ce6a4edab7ce722edf0f2988` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:34.523146 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5395b07-7bea-4ec2-97e0-28f16bc40248 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:34.525088 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d677781c-87ed-40d8-9246-54081d7158e2 None admin] RBAC: Authorizing `identity:create_grant(project_id=4a4ff416931141ce83908e05cc434590, group_id=f2aa80870e8543bab7b8913e4d9de132, role_id=d1691322ce6a4edab7ce722edf0f2988)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:34.531517 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d677781c-87ed-40d8-9246-54081d7158e2 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:34.565956 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 202/393] 10.222.0.26 () {58 vars in 1442 bytes} [Thu Jul 14 09:04:34 2022] PUT /identity/v3/projects/4a4ff416931141ce83908e05cc434590/groups/f2aa80870e8543bab7b8913e4d9de132/roles/d1691322ce6a4edab7ce722edf0f2988 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:04:34.645454 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5395b07-7bea-4ec2-97e0-28f16bc40248 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:34.682059 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744d7a3f-493c-4ace-85b9-4e7db2e0d282 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:34.682967 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744d7a3f-493c-4ace-85b9-4e7db2e0d282 None admin] REQUEST_METHOD: `PUT` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:34.683277 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744d7a3f-493c-4ace-85b9-4e7db2e0d282 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:34.683555 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744d7a3f-493c-4ace-85b9-4e7db2e0d282 None admin] PATH_INFO: `/v3/projects/4a4ff416931141ce83908e05cc434590/groups/f2aa80870e8543bab7b8913e4d9de132/roles/c4bab4a02e5c4f5f97a024e844a0c072` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:34.685810 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744d7a3f-493c-4ace-85b9-4e7db2e0d282 None admin] RBAC: Authorizing `identity:create_grant(project_id=4a4ff416931141ce83908e05cc434590, group_id=f2aa80870e8543bab7b8913e4d9de132, role_id=c4bab4a02e5c4f5f97a024e844a0c072)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:34.687277 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744d7a3f-493c-4ace-85b9-4e7db2e0d282 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:34.704049 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 192/394] 10.222.0.26 () {58 vars in 1442 bytes} [Thu Jul 14 09:04:34 2022] PUT /identity/v3/projects/4a4ff416931141ce83908e05cc434590/groups/f2aa80870e8543bab7b8913e4d9de132/roles/c4bab4a02e5c4f5f97a024e844a0c072 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:04:34.722297 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5b628b-1d13-43f4-b885-b96e679cdd07 None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:34.722719 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5b628b-1d13-43f4-b885-b96e679cdd07 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:34.723086 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5b628b-1d13-43f4-b885-b96e679cdd07 None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:34.725006 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 203/395] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:04:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:04:34.730555 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d49463-6119-4cb4-9d3f-ef8e85cbfe18 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:34.731075 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d49463-6119-4cb4-9d3f-ef8e85cbfe18 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:34.731537 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d49463-6119-4cb4-9d3f-ef8e85cbfe18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:34.754704 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-f5d49463-6119-4cb4-9d3f-ef8e85cbfe18 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:04:34.756042 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-f5d49463-6119-4cb4-9d3f-ef8e85cbfe18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:04:34.797722 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:34.847272 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 193/396] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:04:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 09:04:34.872378 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5b628b-1d13-43f4-b885-b96e679cdd07 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:34.963689 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:34.991725 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5b628b-1d13-43f4-b885-b96e679cdd07 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:35.027085 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18bb597-4667-4771-9955-32d1aef6e825 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:35.028284 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18bb597-4667-4771-9955-32d1aef6e825 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:35.028576 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18bb597-4667-4771-9955-32d1aef6e825 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:35.028856 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18bb597-4667-4771-9955-32d1aef6e825 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:35.040766 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18bb597-4667-4771-9955-32d1aef6e825 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 09:04:35.048891 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-c18bb597-4667-4771-9955-32d1aef6e825 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 09:04:35.048891 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:04:35.048891 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:04:35.048891 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:04:35.048891 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:04:35.048891 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:04:35.048891 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:04:35.048891 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:04:35.048891 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:04:35.048891 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:04:35.048891 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:04:35.048891 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:04:35.048891 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 09:04:35.048891 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 09:04:35.048891 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 09:04:35.048891 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 09:04:35.049776 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 09:04:35.049776 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 09:04:35.049776 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 09:04:35.049776 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 09:04:35.049776 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:04:35.049776 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:04:35.049776 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:04:35.049776 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:04:35.049776 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:04:35.049776 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:04:35.049776 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:04:35.049776 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 14 09:04:35.049776 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:04:35.049776 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:04:35.049776 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:04:35.049776 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:04:35.050636 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:04:35.050636 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:04:35.050636 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:04:35.050636 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:04:35.050636 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 09:04:35.050636 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 09:04:35.050636 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 09:04:35.050636 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 09:04:35.050636 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 09:04:35.050636 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 09:04:35.050636 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 09:04:35.050636 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 14 09:04:35.050636 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 204/397] 10.222.0.26 () {56 vars in 1099 bytes} [Thu Jul 14 09:04:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:04:35.056035 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d49463-6119-4cb4-9d3f-ef8e85cbfe18 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:35.171613 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d49463-6119-4cb4-9d3f-ef8e85cbfe18 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:35.221475 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6fe5cd-ee53-4382-a6af-5516c224ce6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:35.222072 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6fe5cd-ee53-4382-a6af-5516c224ce6e None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:35.222361 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6fe5cd-ee53-4382-a6af-5516c224ce6e None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:35.222681 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6fe5cd-ee53-4382-a6af-5516c224ce6e None admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:35.223473 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6fe5cd-ee53-4382-a6af-5516c224ce6e None admin] RBAC: Adding query filter params (name=member) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:04:35.223786 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6fe5cd-ee53-4382-a6af-5516c224ce6e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:35.225174 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6fe5cd-ee53-4382-a6af-5516c224ce6e None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:35.239329 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 194/398] 10.222.0.26 () {56 vars in 1101 bytes} [Thu Jul 14 09:04:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:04:35.244810 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18bb597-4667-4771-9955-32d1aef6e825 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:35.361006 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18bb597-4667-4771-9955-32d1aef6e825 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:35.395391 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb02bcc7-5545-4320-b20b-180203b8fb24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:35.396314 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb02bcc7-5545-4320-b20b-180203b8fb24 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:35.396883 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb02bcc7-5545-4320-b20b-180203b8fb24 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:35.397169 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb02bcc7-5545-4320-b20b-180203b8fb24 None admin] PATH_INFO: `/v3/groups/f2aa80870e8543bab7b8913e4d9de132` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:35.398462 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb02bcc7-5545-4320-b20b-180203b8fb24 None admin] RBAC: Authorizing `identity:get_group(group_id=f2aa80870e8543bab7b8913e4d9de132)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:35.399894 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb02bcc7-5545-4320-b20b-180203b8fb24 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:35.401544 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 205/399] 10.222.0.26 () {56 vars in 1180 bytes} [Thu Jul 14 09:04:35 2022] GET /identity/v3/groups/f2aa80870e8543bab7b8913e4d9de132 => generated 226 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:04:35.407009 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6fe5cd-ee53-4382-a6af-5516c224ce6e None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:35.435766 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:35.523805 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6fe5cd-ee53-4382-a6af-5516c224ce6e None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:35.573350 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691d5bfb-4007-4a68-b7c2-d012853364b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:35.574226 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691d5bfb-4007-4a68-b7c2-d012853364b7 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:35.574508 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691d5bfb-4007-4a68-b7c2-d012853364b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:35.574781 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691d5bfb-4007-4a68-b7c2-d012853364b7 None admin] PATH_INFO: `/v3/projects/a596189576264bfc9638ca33f184759c` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:35.580033 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691d5bfb-4007-4a68-b7c2-d012853364b7 None admin] RBAC: Authorizing `identity:get_project(project_id=a596189576264bfc9638ca33f184759c)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:35.581513 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691d5bfb-4007-4a68-b7c2-d012853364b7 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:35.583681 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 195/400] 10.222.0.26 () {56 vars in 1186 bytes} [Thu Jul 14 09:04:35 2022] GET /identity/v3/projects/a596189576264bfc9638ca33f184759c => generated 302 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:04:35.589272 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb02bcc7-5545-4320-b20b-180203b8fb24 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:35.592237 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:35.702177 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb02bcc7-5545-4320-b20b-180203b8fb24 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:35.737163 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991b30ad-61b1-4bef-80af-bf5b52608f0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:35.738066 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991b30ad-61b1-4bef-80af-bf5b52608f0c None admin] REQUEST_METHOD: `PUT` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:35.738351 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991b30ad-61b1-4bef-80af-bf5b52608f0c None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:35.738621 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991b30ad-61b1-4bef-80af-bf5b52608f0c None admin] PATH_INFO: `/v3/projects/a596189576264bfc9638ca33f184759c/groups/f2aa80870e8543bab7b8913e4d9de132/roles/d1691322ce6a4edab7ce722edf0f2988` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:35.740711 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991b30ad-61b1-4bef-80af-bf5b52608f0c None admin] RBAC: Authorizing `identity:create_grant(project_id=a596189576264bfc9638ca33f184759c, group_id=f2aa80870e8543bab7b8913e4d9de132, role_id=d1691322ce6a4edab7ce722edf0f2988)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:35.742218 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991b30ad-61b1-4bef-80af-bf5b52608f0c None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:35.757599 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 206/401] 10.222.0.26 () {58 vars in 1442 bytes} [Thu Jul 14 09:04:35 2022] PUT /identity/v3/projects/a596189576264bfc9638ca33f184759c/groups/f2aa80870e8543bab7b8913e4d9de132/roles/d1691322ce6a4edab7ce722edf0f2988 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:04:35.973070 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf2b86c-4364-4894-9034-c1c33fdfb10a None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:35.973578 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf2b86c-4364-4894-9034-c1c33fdfb10a None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:35.973659 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf2b86c-4364-4894-9034-c1c33fdfb10a None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:35.975281 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 196/402] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:04:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:04:35.980775 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ad6f0e-41ee-4d7b-8b63-8900647fd7d2 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:35.981203 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ad6f0e-41ee-4d7b-8b63-8900647fd7d2 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:35.981398 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ad6f0e-41ee-4d7b-8b63-8900647fd7d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:35.999296 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-83ad6f0e-41ee-4d7b-8b63-8900647fd7d2 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:04:36.000493 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-83ad6f0e-41ee-4d7b-8b63-8900647fd7d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:04:36.065850 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:36.077615 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 207/403] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:04:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 09:04:36.101498 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf2b86c-4364-4894-9034-c1c33fdfb10a None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:36.219398 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:36.230989 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf2b86c-4364-4894-9034-c1c33fdfb10a None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:36.274906 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bbfe56-6cf2-4bb7-9733-262b46388e44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:36.275761 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bbfe56-6cf2-4bb7-9733-262b46388e44 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:36.276083 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bbfe56-6cf2-4bb7-9733-262b46388e44 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:36.276361 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bbfe56-6cf2-4bb7-9733-262b46388e44 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:36.289427 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bbfe56-6cf2-4bb7-9733-262b46388e44 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 09:04:36.299900 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-a3bbfe56-6cf2-4bb7-9733-262b46388e44 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 09:04:36.299900 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:04:36.299900 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:04:36.299900 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:04:36.299900 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:04:36.299900 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:04:36.299900 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:04:36.299900 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:04:36.299900 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:04:36.299900 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:04:36.299900 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:04:36.299900 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:04:36.299900 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 09:04:36.299900 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 09:04:36.299900 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 09:04:36.299900 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 09:04:36.300797 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 09:04:36.300797 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 09:04:36.300797 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 09:04:36.300797 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 09:04:36.300797 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:04:36.300797 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:04:36.300797 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:04:36.300797 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:04:36.300797 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:04:36.300797 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:04:36.300797 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:04:36.300797 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 14 09:04:36.300797 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:04:36.300797 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:04:36.300797 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:04:36.300797 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:04:36.301599 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:04:36.301599 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:04:36.301599 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:04:36.301599 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:04:36.301599 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 09:04:36.301599 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 09:04:36.301599 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 09:04:36.301599 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 09:04:36.301599 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 09:04:36.301599 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 09:04:36.301599 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 09:04:36.301599 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 14 09:04:36.302203 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 197/404] 10.222.0.26 () {56 vars in 1114 bytes} [Thu Jul 14 09:04:36 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:04:36.307958 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ad6f0e-41ee-4d7b-8b63-8900647fd7d2 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:36.411348 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ad6f0e-41ee-4d7b-8b63-8900647fd7d2 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:36.446516 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bc3845-896a-407a-bdc5-eb481b8d3b1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:36.447277 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bc3845-896a-407a-bdc5-eb481b8d3b1a None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:36.447937 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bc3845-896a-407a-bdc5-eb481b8d3b1a None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:36.448218 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bc3845-896a-407a-bdc5-eb481b8d3b1a None admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:36.448988 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63bc3845-896a-407a-bdc5-eb481b8d3b1a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:04:36.449297 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63bc3845-896a-407a-bdc5-eb481b8d3b1a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:36.450687 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63bc3845-896a-407a-bdc5-eb481b8d3b1a None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:36.459903 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 208/405] 10.222.0.26 () {56 vars in 1111 bytes} [Thu Jul 14 09:04:36 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:04:36.465471 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bbfe56-6cf2-4bb7-9733-262b46388e44 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:36.574909 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bbfe56-6cf2-4bb7-9733-262b46388e44 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:36.602988 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e11d29-dc82-40a6-b3df-38a635e5c15b None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:36.605366 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e11d29-dc82-40a6-b3df-38a635e5c15b None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:36.605995 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e11d29-dc82-40a6-b3df-38a635e5c15b None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:36.607866 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 209/406] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:04:36 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:04:36.616054 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e973ad-bf3e-4602-a2ce-ef2bc00bb04f None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:36.616701 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e973ad-bf3e-4602-a2ce-ef2bc00bb04f None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:36.617307 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e973ad-bf3e-4602-a2ce-ef2bc00bb04f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:36.617617 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23ff7f6-9bbf-488e-bb6b-29fedd0b13dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:36.618476 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23ff7f6-9bbf-488e-bb6b-29fedd0b13dd None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:36.618757 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23ff7f6-9bbf-488e-bb6b-29fedd0b13dd None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:36.619032 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23ff7f6-9bbf-488e-bb6b-29fedd0b13dd None admin] PATH_INFO: `/v3/groups/f2aa80870e8543bab7b8913e4d9de132` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:36.620625 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23ff7f6-9bbf-488e-bb6b-29fedd0b13dd None admin] RBAC: Authorizing `identity:get_group(group_id=f2aa80870e8543bab7b8913e4d9de132)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:36.622067 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23ff7f6-9bbf-488e-bb6b-29fedd0b13dd None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:36.624016 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 198/407] 10.222.0.26 () {56 vars in 1180 bytes} [Thu Jul 14 09:04:36 2022] GET /identity/v3/groups/f2aa80870e8543bab7b8913e4d9de132 => generated 226 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:04:36.635234 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23ff7f6-9bbf-488e-bb6b-29fedd0b13dd None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:36.652792 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-89e973ad-bf3e-4602-a2ce-ef2bc00bb04f None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:04:36.656124 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-89e973ad-bf3e-4602-a2ce-ef2bc00bb04f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:04:36.680865 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:36.768070 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:36.796021 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 210/408] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:04:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 09:04:36.800307 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23ff7f6-9bbf-488e-bb6b-29fedd0b13dd None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:36.828357 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e973ad-bf3e-4602-a2ce-ef2bc00bb04f None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:36.852702 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d44595-ed4b-460c-9749-de96e5a22559 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:36.853573 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d44595-ed4b-460c-9749-de96e5a22559 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:36.853857 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d44595-ed4b-460c-9749-de96e5a22559 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:36.854137 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d44595-ed4b-460c-9749-de96e5a22559 None admin] PATH_INFO: `/v3/projects/a596189576264bfc9638ca33f184759c` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:36.856162 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d44595-ed4b-460c-9749-de96e5a22559 None admin] RBAC: Authorizing `identity:get_project(project_id=a596189576264bfc9638ca33f184759c)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:36.857619 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d44595-ed4b-460c-9749-de96e5a22559 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:36.859676 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 199/409] 10.222.0.26 () {56 vars in 1186 bytes} [Thu Jul 14 09:04:36 2022] GET /identity/v3/projects/a596189576264bfc9638ca33f184759c => generated 302 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:04:36.865398 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d44595-ed4b-460c-9749-de96e5a22559 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:36.966572 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:36.979893 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e973ad-bf3e-4602-a2ce-ef2bc00bb04f None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:36.983152 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d44595-ed4b-460c-9749-de96e5a22559 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:37.018855 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85be742-632c-42d0-bbbb-6b8f15f4b9c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:37.019781 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85be742-632c-42d0-bbbb-6b8f15f4b9c3 None admin] REQUEST_METHOD: `PUT` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:37.020077 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85be742-632c-42d0-bbbb-6b8f15f4b9c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:37.020357 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85be742-632c-42d0-bbbb-6b8f15f4b9c3 None admin] PATH_INFO: `/v3/projects/a596189576264bfc9638ca33f184759c/groups/f2aa80870e8543bab7b8913e4d9de132/roles/c4bab4a02e5c4f5f97a024e844a0c072` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:37.022722 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85be742-632c-42d0-bbbb-6b8f15f4b9c3 None admin] RBAC: Authorizing `identity:create_grant(project_id=a596189576264bfc9638ca33f184759c, group_id=f2aa80870e8543bab7b8913e4d9de132, role_id=c4bab4a02e5c4f5f97a024e844a0c072)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:37.024241 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85be742-632c-42d0-bbbb-6b8f15f4b9c3 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:37.034229 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7085deb-1087-4c01-9ad7-f55f208043a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:37.035181 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7085deb-1087-4c01-9ad7-f55f208043a8 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:37.035488 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7085deb-1087-4c01-9ad7-f55f208043a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:37.035856 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7085deb-1087-4c01-9ad7-f55f208043a8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:37.045070 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 200/410] 10.222.0.26 () {58 vars in 1442 bytes} [Thu Jul 14 09:04:36 2022] PUT /identity/v3/projects/a596189576264bfc9638ca33f184759c/groups/f2aa80870e8543bab7b8913e4d9de132/roles/c4bab4a02e5c4f5f97a024e844a0c072 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:04:37.059837 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7085deb-1087-4c01-9ad7-f55f208043a8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 09:04:37.069970 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-d7085deb-1087-4c01-9ad7-f55f208043a8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 09:04:37.069970 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:04:37.069970 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:04:37.069970 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:04:37.069970 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:04:37.069970 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:04:37.069970 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:04:37.069970 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:04:37.069970 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:04:37.069970 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:04:37.069970 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:04:37.069970 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:04:37.069970 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 09:04:37.069970 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 09:04:37.069970 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 09:04:37.069970 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 09:04:37.070909 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 09:04:37.070909 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 09:04:37.070909 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 09:04:37.070909 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 09:04:37.070909 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:04:37.070909 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:04:37.070909 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:04:37.070909 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:04:37.070909 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:04:37.070909 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:04:37.070909 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:04:37.070909 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 14 09:04:37.070909 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:04:37.070909 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:04:37.070909 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:04:37.070909 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:04:37.071807 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:04:37.071807 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:04:37.071807 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:04:37.071807 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:04:37.071807 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 09:04:37.071807 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 09:04:37.071807 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 09:04:37.071807 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 09:04:37.071807 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 09:04:37.071807 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 09:04:37.071807 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 09:04:37.071807 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 14 09:04:37.071807 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 211/411] 10.222.0.26 () {56 vars in 1096 bytes} [Thu Jul 14 09:04:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 09:04:37.077325 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85be742-632c-42d0-bbbb-6b8f15f4b9c3 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:37.089383 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675ddb53-3dde-4593-87a4-2368cf94b64e None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:37.089670 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675ddb53-3dde-4593-87a4-2368cf94b64e None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:37.089830 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675ddb53-3dde-4593-87a4-2368cf94b64e None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:37.093913 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 212/412] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:04:37 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:04:37.102290 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6454e63-f8e3-4f91-b471-a775e09f6259 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:37.102820 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6454e63-f8e3-4f91-b471-a775e09f6259 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:37.103668 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6454e63-f8e3-4f91-b471-a775e09f6259 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:37.137061 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-c6454e63-f8e3-4f91-b471-a775e09f6259 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:04:37.140294 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-c6454e63-f8e3-4f91-b471-a775e09f6259 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:04:37.226880 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:37.267882 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85be742-632c-42d0-bbbb-6b8f15f4b9c3 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:37.277158 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 213/413] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:04:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 09:04:37.301116 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6454e63-f8e3-4f91-b471-a775e09f6259 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:37.308007 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe5518f-07c2-413d-bb89-4dba011d01a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:37.308950 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe5518f-07c2-413d-bb89-4dba011d01a9 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:37.309389 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe5518f-07c2-413d-bb89-4dba011d01a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:37.309804 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe5518f-07c2-413d-bb89-4dba011d01a9 None admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:37.311248 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe5518f-07c2-413d-bb89-4dba011d01a9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:04:37.311737 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe5518f-07c2-413d-bb89-4dba011d01a9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:37.314221 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe5518f-07c2-413d-bb89-4dba011d01a9 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:37.325653 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 201/414] 10.222.0.26 () {56 vars in 1099 bytes} [Thu Jul 14 09:04:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:04:37.333146 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe5518f-07c2-413d-bb89-4dba011d01a9 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:37.349846 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:37.433167 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6454e63-f8e3-4f91-b471-a775e09f6259 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:37.498840 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fad8e3-e96a-4b5b-b06d-1b60887e7e6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:37.499878 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fad8e3-e96a-4b5b-b06d-1b60887e7e6a None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:37.500153 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fad8e3-e96a-4b5b-b06d-1b60887e7e6a None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:37.500424 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fad8e3-e96a-4b5b-b06d-1b60887e7e6a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:37.520586 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fad8e3-e96a-4b5b-b06d-1b60887e7e6a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 09:04:37.532726 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe5518f-07c2-413d-bb89-4dba011d01a9 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:37.535512 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-c3fad8e3-e96a-4b5b-b06d-1b60887e7e6a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 09:04:37.535512 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:04:37.535512 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:04:37.535512 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:04:37.535512 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:04:37.535512 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:04:37.535512 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:04:37.535512 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:04:37.535512 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:04:37.535512 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:04:37.535512 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:04:37.535512 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:04:37.535512 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 09:04:37.535512 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 09:04:37.535512 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 09:04:37.535512 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 09:04:37.536409 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 09:04:37.536409 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 09:04:37.536409 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 09:04:37.536409 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 09:04:37.536409 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:04:37.536409 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:04:37.536409 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:04:37.536409 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:04:37.536409 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:04:37.536409 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:04:37.536409 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:04:37.536409 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 14 09:04:37.536409 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:04:37.536409 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:04:37.536409 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:04:37.536409 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:04:37.537232 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:04:37.537232 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:04:37.537232 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:04:37.537232 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:04:37.537232 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 09:04:37.537232 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 09:04:37.537232 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 09:04:37.537232 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 09:04:37.537232 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 09:04:37.537232 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 09:04:37.537232 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 09:04:37.537232 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 14 09:04:37.541688 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 214/415] 10.222.0.26 () {56 vars in 1099 bytes} [Thu Jul 14 09:04:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:04:37.549574 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fad8e3-e96a-4b5b-b06d-1b60887e7e6a None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:37.583390 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ed2441-bf13-4fb5-9fd5-428e33522813 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:37.584418 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:37.584732 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ed2441-bf13-4fb5-9fd5-428e33522813 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:37.585283 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ed2441-bf13-4fb5-9fd5-428e33522813 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:37.585504 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ed2441-bf13-4fb5-9fd5-428e33522813 None admin] PATH_INFO: `/v3/groups/ae3d183691c04d43b40da20c4381678e` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:37.587843 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ed2441-bf13-4fb5-9fd5-428e33522813 None admin] RBAC: Authorizing `identity:get_group(group_id=ae3d183691c04d43b40da20c4381678e)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:37.589420 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ed2441-bf13-4fb5-9fd5-428e33522813 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:37.591233 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 202/416] 10.222.0.26 () {56 vars in 1180 bytes} [Thu Jul 14 09:04:37 2022] GET /identity/v3/groups/ae3d183691c04d43b40da20c4381678e => generated 229 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:04:37.592370 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceeccf21-53e7-4752-b846-f583cee00b33 None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:37.595310 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceeccf21-53e7-4752-b846-f583cee00b33 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:37.595663 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceeccf21-53e7-4752-b846-f583cee00b33 None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:37.596970 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 203/417] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:04:37 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:04:37.602156 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceeccf21-53e7-4752-b846-f583cee00b33 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:37.677736 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fad8e3-e96a-4b5b-b06d-1b60887e7e6a None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:37.693788 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:37.717740 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87b333c-7cf2-402a-8c08-2d4abfb73137 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:37.718542 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87b333c-7cf2-402a-8c08-2d4abfb73137 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:37.718924 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87b333c-7cf2-402a-8c08-2d4abfb73137 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:37.719362 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87b333c-7cf2-402a-8c08-2d4abfb73137 None admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:37.720340 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87b333c-7cf2-402a-8c08-2d4abfb73137 None admin] RBAC: Adding query filter params (name=member) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:04:37.720747 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87b333c-7cf2-402a-8c08-2d4abfb73137 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:37.722199 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceeccf21-53e7-4752-b846-f583cee00b33 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:37.722326 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87b333c-7cf2-402a-8c08-2d4abfb73137 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:37.736246 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 215/418] 10.222.0.26 () {56 vars in 1101 bytes} [Thu Jul 14 09:04:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:04:37.736246 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7aaffad-4fc9-4687-b38c-baa689736912 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:37.736246 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7aaffad-4fc9-4687-b38c-baa689736912 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:37.736246 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7aaffad-4fc9-4687-b38c-baa689736912 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:37.759266 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-f7aaffad-4fc9-4687-b38c-baa689736912 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:04:37.761527 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-f7aaffad-4fc9-4687-b38c-baa689736912 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:04:37.769817 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85fd0dc-6783-45dd-9318-c5a7b9cdd706 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:37.770818 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85fd0dc-6783-45dd-9318-c5a7b9cdd706 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:37.771231 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85fd0dc-6783-45dd-9318-c5a7b9cdd706 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:37.771603 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85fd0dc-6783-45dd-9318-c5a7b9cdd706 None admin] PATH_INFO: `/v3/projects/2fd144ddf49f41d58d6674bed415082b` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:37.773298 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85fd0dc-6783-45dd-9318-c5a7b9cdd706 None admin] RBAC: Authorizing `identity:get_project(project_id=2fd144ddf49f41d58d6674bed415082b)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:37.775040 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85fd0dc-6783-45dd-9318-c5a7b9cdd706 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:37.781578 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 204/419] 10.222.0.26 () {56 vars in 1186 bytes} [Thu Jul 14 09:04:37 2022] GET /identity/v3/projects/2fd144ddf49f41d58d6674bed415082b => generated 344 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:04:37.781578 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85fd0dc-6783-45dd-9318-c5a7b9cdd706 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:37.850672 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 216/420] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:04:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 09:04:37.851714 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7aaffad-4fc9-4687-b38c-baa689736912 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:37.896521 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:37.903259 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85fd0dc-6783-45dd-9318-c5a7b9cdd706 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:37.952612 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98340ef3-12fb-4a6c-9049-ea163b6c5cd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:37.952849 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98340ef3-12fb-4a6c-9049-ea163b6c5cd4 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:37.953139 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98340ef3-12fb-4a6c-9049-ea163b6c5cd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:37.953415 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98340ef3-12fb-4a6c-9049-ea163b6c5cd4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:37.970708 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:37.993421 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7aaffad-4fc9-4687-b38c-baa689736912 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:38.029648 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98340ef3-12fb-4a6c-9049-ea163b6c5cd4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:38.031142 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98340ef3-12fb-4a6c-9049-ea163b6c5cd4 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:38.038452 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84c36dc-ca8c-4fea-8b2a-f925a81d7908 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:38.040451 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84c36dc-ca8c-4fea-8b2a-f925a81d7908 None admin] REQUEST_METHOD: `PUT` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:38.040930 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84c36dc-ca8c-4fea-8b2a-f925a81d7908 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:38.041826 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84c36dc-ca8c-4fea-8b2a-f925a81d7908 None admin] PATH_INFO: `/v3/projects/2fd144ddf49f41d58d6674bed415082b/groups/ae3d183691c04d43b40da20c4381678e/roles/7b549d58d1724395b343609d895c80d0` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:38.044713 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84c36dc-ca8c-4fea-8b2a-f925a81d7908 None admin] RBAC: Authorizing `identity:create_grant(project_id=2fd144ddf49f41d58d6674bed415082b, group_id=ae3d183691c04d43b40da20c4381678e, role_id=7b549d58d1724395b343609d895c80d0)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:38.047388 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84c36dc-ca8c-4fea-8b2a-f925a81d7908 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:38.071388 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 217/421] 10.222.0.26 () {58 vars in 1442 bytes} [Thu Jul 14 09:04:37 2022] PUT /identity/v3/projects/2fd144ddf49f41d58d6674bed415082b/groups/ae3d183691c04d43b40da20c4381678e/roles/7b549d58d1724395b343609d895c80d0 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:04:38.073737 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84c36dc-ca8c-4fea-8b2a-f925a81d7908 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:38.211953 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 205/422] 10.222.0.26 () {58 vars in 1282 bytes} [Thu Jul 14 09:04:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 09:04:38.220483 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98340ef3-12fb-4a6c-9049-ea163b6c5cd4 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:38.248185 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:38.271905 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84c36dc-ca8c-4fea-8b2a-f925a81d7908 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:38.289662 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:38.311554 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b00bb2c-923b-42c8-95db-b34898d006b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:38.312337 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b00bb2c-923b-42c8-95db-b34898d006b3 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:38.312620 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b00bb2c-923b-42c8-95db-b34898d006b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:38.312894 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b00bb2c-923b-42c8-95db-b34898d006b3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:38.324194 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b00bb2c-923b-42c8-95db-b34898d006b3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 09:04:38.334674 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-8b00bb2c-923b-42c8-95db-b34898d006b3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 09:04:38.334674 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:04:38.334674 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:04:38.334674 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:04:38.334674 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:04:38.334674 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:04:38.334674 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:04:38.334674 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:04:38.334674 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:04:38.334674 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:04:38.334674 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:04:38.334674 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:04:38.334674 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 09:04:38.334674 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 09:04:38.334674 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 09:04:38.334674 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 09:04:38.335853 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 09:04:38.335853 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 09:04:38.335853 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 09:04:38.335853 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 09:04:38.335853 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:04:38.335853 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:04:38.335853 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:04:38.335853 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:04:38.335853 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:04:38.335853 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:04:38.335853 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:04:38.335853 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 14 09:04:38.335853 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:04:38.335853 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:04:38.335853 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:04:38.335853 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:04:38.337834 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:04:38.337834 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:04:38.337834 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:04:38.337834 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:04:38.337834 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 09:04:38.337834 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 09:04:38.337834 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 09:04:38.337834 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 09:04:38.337834 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 09:04:38.337834 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 09:04:38.337834 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 09:04:38.337834 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 14 09:04:38.337834 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 218/423] 10.222.0.26 () {56 vars in 1114 bytes} [Thu Jul 14 09:04:38 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:04:38.342902 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b00bb2c-923b-42c8-95db-b34898d006b3 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:38.354603 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98340ef3-12fb-4a6c-9049-ea163b6c5cd4 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:38.390473 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da04def2-9a45-495b-a1bc-ad562122d872 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:38.391882 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da04def2-9a45-495b-a1bc-ad562122d872 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:38.392341 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da04def2-9a45-495b-a1bc-ad562122d872 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:38.392847 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da04def2-9a45-495b-a1bc-ad562122d872 None admin] PATH_INFO: `/v3/projects/4a4ff416931141ce83908e05cc434590` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:38.394329 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da04def2-9a45-495b-a1bc-ad562122d872 None admin] RBAC: Authorizing `identity:get_project(project_id=4a4ff416931141ce83908e05cc434590)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:38.396097 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da04def2-9a45-495b-a1bc-ad562122d872 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:38.398456 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 206/424] 10.222.0.26 () {56 vars in 1186 bytes} [Thu Jul 14 09:04:38 2022] GET /identity/v3/projects/4a4ff416931141ce83908e05cc434590 => generated 298 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:04:38.400022 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2e9a20-413b-4273-9067-1bf7c1dfe176 None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:38.400483 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2e9a20-413b-4273-9067-1bf7c1dfe176 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:38.400945 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2e9a20-413b-4273-9067-1bf7c1dfe176 None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:38.402701 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 207/425] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:04:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:04:38.409326 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb0a2c9-5969-4c06-bf65-8e60548bd8e2 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:38.409863 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb0a2c9-5969-4c06-bf65-8e60548bd8e2 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:38.410143 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb0a2c9-5969-4c06-bf65-8e60548bd8e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:38.427495 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-9eb0a2c9-5969-4c06-bf65-8e60548bd8e2 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:04:38.428701 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-9eb0a2c9-5969-4c06-bf65-8e60548bd8e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:04:38.450275 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b00bb2c-923b-42c8-95db-b34898d006b3 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:38.486941 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09036ba3-9569-4a23-9cd1-608a0d03aa8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:38.488084 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09036ba3-9569-4a23-9cd1-608a0d03aa8b None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:38.488709 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09036ba3-9569-4a23-9cd1-608a0d03aa8b None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:38.489209 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09036ba3-9569-4a23-9cd1-608a0d03aa8b None admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:38.490245 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09036ba3-9569-4a23-9cd1-608a0d03aa8b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:04:38.490865 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09036ba3-9569-4a23-9cd1-608a0d03aa8b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:38.492749 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09036ba3-9569-4a23-9cd1-608a0d03aa8b None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:38.496344 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 208/426] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:04:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 09:04:38.497266 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb0a2c9-5969-4c06-bf65-8e60548bd8e2 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:38.513568 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 219/427] 10.222.0.26 () {56 vars in 1111 bytes} [Thu Jul 14 09:04:38 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:04:38.519617 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09036ba3-9569-4a23-9cd1-608a0d03aa8b None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:38.535281 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:38.554197 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:38.608636 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb0a2c9-5969-4c06-bf65-8e60548bd8e2 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:38.631288 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09036ba3-9569-4a23-9cd1-608a0d03aa8b None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:38.645007 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047d34df-f69e-481e-b2d1-48059a67a371 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:38.646051 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047d34df-f69e-481e-b2d1-48059a67a371 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:38.646526 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047d34df-f69e-481e-b2d1-48059a67a371 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:38.646981 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047d34df-f69e-481e-b2d1-48059a67a371 None admin] PATH_INFO: `/v3/groups/f2aa80870e8543bab7b8913e4d9de132` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:38.648359 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047d34df-f69e-481e-b2d1-48059a67a371 None admin] RBAC: Authorizing `identity:get_group(group_id=f2aa80870e8543bab7b8913e4d9de132)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:38.649989 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047d34df-f69e-481e-b2d1-48059a67a371 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:38.651888 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 209/428] 10.222.0.26 () {56 vars in 1180 bytes} [Thu Jul 14 09:04:38 2022] GET /identity/v3/groups/f2aa80870e8543bab7b8913e4d9de132 => generated 226 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:04:38.653041 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047d34df-f69e-481e-b2d1-48059a67a371 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:38.666714 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f87910-e2ef-49be-aed9-b5e34bb5d30c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:38.667650 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f87910-e2ef-49be-aed9-b5e34bb5d30c None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:38.668044 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f87910-e2ef-49be-aed9-b5e34bb5d30c None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:38.668437 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f87910-e2ef-49be-aed9-b5e34bb5d30c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:38.679917 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f87910-e2ef-49be-aed9-b5e34bb5d30c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 09:04:38.688419 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-a3f87910-e2ef-49be-aed9-b5e34bb5d30c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 09:04:38.688419 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:04:38.688419 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:04:38.688419 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:04:38.688419 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:04:38.688419 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:04:38.688419 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:04:38.688419 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:04:38.688419 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:04:38.688419 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:04:38.688419 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:04:38.688419 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:04:38.688419 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 09:04:38.688419 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 09:04:38.688419 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 09:04:38.688419 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 09:04:38.689338 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 09:04:38.689338 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 09:04:38.689338 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 09:04:38.689338 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 09:04:38.689338 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:04:38.689338 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:04:38.689338 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:04:38.689338 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:04:38.689338 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:04:38.689338 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:04:38.689338 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:04:38.689338 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 14 09:04:38.689338 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:04:38.689338 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:04:38.689338 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:04:38.689338 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:04:38.690344 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:04:38.690344 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:04:38.690344 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:04:38.690344 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:04:38.690344 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 09:04:38.690344 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 09:04:38.690344 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 09:04:38.690344 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 09:04:38.690344 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 09:04:38.690344 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 09:04:38.690344 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 09:04:38.690344 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 14 09:04:38.690344 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 220/429] 10.222.0.26 () {56 vars in 1099 bytes} [Thu Jul 14 09:04:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 09:04:38.691244 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f87910-e2ef-49be-aed9-b5e34bb5d30c None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:38.761091 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047d34df-f69e-481e-b2d1-48059a67a371 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:38.771605 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:38.774950 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:38.798001 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67afb82-0757-436a-9b81-98894a000191 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:38.798968 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67afb82-0757-436a-9b81-98894a000191 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:38.799449 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67afb82-0757-436a-9b81-98894a000191 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:38.799867 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f87910-e2ef-49be-aed9-b5e34bb5d30c None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:38.800074 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67afb82-0757-436a-9b81-98894a000191 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:38.836714 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd34881d-9177-488c-99da-04fe3352ed3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:38.837486 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd34881d-9177-488c-99da-04fe3352ed3d None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:38.837836 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd34881d-9177-488c-99da-04fe3352ed3d None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:38.838181 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd34881d-9177-488c-99da-04fe3352ed3d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:38.839086 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd34881d-9177-488c-99da-04fe3352ed3d None admin] RBAC: Adding query filter params (group.id=f2aa80870e8543bab7b8913e4d9de132, role.id=d1691322ce6a4edab7ce722edf0f2988, scope.project.id=4a4ff416931141ce83908e05cc434590) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:04:38.839492 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd34881d-9177-488c-99da-04fe3352ed3d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:38.841019 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd34881d-9177-488c-99da-04fe3352ed3d None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:38.847710 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 221/430] 10.222.0.26 () {56 vars in 1376 bytes} [Thu Jul 14 09:04:38 2022] GET /identity/v3/role_assignments?group.id=f2aa80870e8543bab7b8913e4d9de132&scope.project.id=4a4ff416931141ce83908e05cc434590&role.id=d1691322ce6a4edab7ce722edf0f2988 => generated 611 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:04:38.848420 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd34881d-9177-488c-99da-04fe3352ed3d None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:38.874115 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67afb82-0757-436a-9b81-98894a000191 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:38.876235 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67afb82-0757-436a-9b81-98894a000191 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:38.963914 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd34881d-9177-488c-99da-04fe3352ed3d None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:38.985613 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 210/431] 10.222.0.26 () {58 vars in 1282 bytes} [Thu Jul 14 09:04:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 09:04:38.993283 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67afb82-0757-436a-9b81-98894a000191 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:39.007013 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6dcd62-ee24-463d-838a-11849aa2bea5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:39.007827 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6dcd62-ee24-463d-838a-11849aa2bea5 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:39.008490 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:39.009614 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6dcd62-ee24-463d-838a-11849aa2bea5 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:39.010235 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6dcd62-ee24-463d-838a-11849aa2bea5 None admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:39.013331 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6dcd62-ee24-463d-838a-11849aa2bea5 None admin] RBAC: Adding query filter params (name=member) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:04:39.013331 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6dcd62-ee24-463d-838a-11849aa2bea5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:39.013331 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6dcd62-ee24-463d-838a-11849aa2bea5 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:39.023130 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 222/432] 10.222.0.26 () {56 vars in 1101 bytes} [Thu Jul 14 09:04:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:04:39.028464 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:39.030264 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6dcd62-ee24-463d-838a-11849aa2bea5 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:39.143327 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6dcd62-ee24-463d-838a-11849aa2bea5 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:39.176096 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67afb82-0757-436a-9b81-98894a000191 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:39.189364 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7465f1-53ba-4ae5-a763-3a6f679659d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:39.190167 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7465f1-53ba-4ae5-a763-3a6f679659d7 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:39.190551 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7465f1-53ba-4ae5-a763-3a6f679659d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:39.190925 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7465f1-53ba-4ae5-a763-3a6f679659d7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:39.219805 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6234a7c0-20b9-4d17-8874-28e944595fb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:39.220682 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6234a7c0-20b9-4d17-8874-28e944595fb5 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:39.221239 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6234a7c0-20b9-4d17-8874-28e944595fb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:39.221521 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6234a7c0-20b9-4d17-8874-28e944595fb5 None admin] PATH_INFO: `/v3/projects/4a4ff416931141ce83908e05cc434590` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:39.223346 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6234a7c0-20b9-4d17-8874-28e944595fb5 None admin] RBAC: Authorizing `identity:get_project(project_id=4a4ff416931141ce83908e05cc434590)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:39.225269 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6234a7c0-20b9-4d17-8874-28e944595fb5 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:39.227972 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 211/433] 10.222.0.26 () {56 vars in 1186 bytes} [Thu Jul 14 09:04:38 2022] GET /identity/v3/projects/4a4ff416931141ce83908e05cc434590 => generated 298 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:04:39.234206 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6234a7c0-20b9-4d17-8874-28e944595fb5 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:39.265134 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:39.277729 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7465f1-53ba-4ae5-a763-3a6f679659d7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:39.279212 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7465f1-53ba-4ae5-a763-3a6f679659d7 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:39.340351 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:39.370502 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6234a7c0-20b9-4d17-8874-28e944595fb5 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:39.405837 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 223/434] 10.222.0.26 () {58 vars in 1282 bytes} [Thu Jul 14 09:04:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 14 09:04:39.411833 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8a40ed-25e5-41bc-a9fc-c6de994d6299 None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:39.412408 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8a40ed-25e5-41bc-a9fc-c6de994d6299 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:39.412911 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8a40ed-25e5-41bc-a9fc-c6de994d6299 None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:39.414448 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 224/435] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:04:39 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:04:39.414983 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc981261-97bc-4bdb-b2d9-627761540553 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:39.415506 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8a40ed-25e5-41bc-a9fc-c6de994d6299 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:39.415920 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc981261-97bc-4bdb-b2d9-627761540553 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:39.416204 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc981261-97bc-4bdb-b2d9-627761540553 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:39.416482 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc981261-97bc-4bdb-b2d9-627761540553 None admin] PATH_INFO: `/v3/groups/f2aa80870e8543bab7b8913e4d9de132` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:39.418900 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc981261-97bc-4bdb-b2d9-627761540553 None admin] RBAC: Authorizing `identity:get_group(group_id=f2aa80870e8543bab7b8913e4d9de132)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:39.420430 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc981261-97bc-4bdb-b2d9-627761540553 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:39.422545 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 212/436] 10.222.0.26 () {56 vars in 1180 bytes} [Thu Jul 14 09:04:39 2022] GET /identity/v3/groups/f2aa80870e8543bab7b8913e4d9de132 => generated 226 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 09:04:39.428165 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdce400-38f4-4349-a175-c50cad0fc33f None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:39.428452 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdce400-38f4-4349-a175-c50cad0fc33f None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:39.428727 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdce400-38f4-4349-a175-c50cad0fc33f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:39.449492 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-4cdce400-38f4-4349-a175-c50cad0fc33f None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:04:39.451047 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-4cdce400-38f4-4349-a175-c50cad0fc33f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:04:39.543145 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 213/437] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:04:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 09:04:39.544393 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdce400-38f4-4349-a175-c50cad0fc33f None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:39.563724 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:39.573545 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8a40ed-25e5-41bc-a9fc-c6de994d6299 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:39.610327 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d9396b-70f1-4a2a-8855-92e3821f3606 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:39.611230 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d9396b-70f1-4a2a-8855-92e3821f3606 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:39.611518 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d9396b-70f1-4a2a-8855-92e3821f3606 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:39.611796 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d9396b-70f1-4a2a-8855-92e3821f3606 None admin] PATH_INFO: `/v3/projects/a596189576264bfc9638ca33f184759c` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:39.613001 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d9396b-70f1-4a2a-8855-92e3821f3606 None admin] RBAC: Authorizing `identity:get_project(project_id=a596189576264bfc9638ca33f184759c)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:39.614450 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d9396b-70f1-4a2a-8855-92e3821f3606 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:39.616229 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 225/438] 10.222.0.26 () {56 vars in 1186 bytes} [Thu Jul 14 09:04:39 2022] GET /identity/v3/projects/a596189576264bfc9638ca33f184759c => generated 302 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:04:39.616990 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d9396b-70f1-4a2a-8855-92e3821f3606 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:39.643919 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:39.666832 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdce400-38f4-4349-a175-c50cad0fc33f None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:39.702977 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7beb5ba-42b8-42e4-be1d-640127ee5fab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:39.703797 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7beb5ba-42b8-42e4-be1d-640127ee5fab None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:39.704081 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7beb5ba-42b8-42e4-be1d-640127ee5fab None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:39.704452 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7beb5ba-42b8-42e4-be1d-640127ee5fab None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:39.705305 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7beb5ba-42b8-42e4-be1d-640127ee5fab None admin] RBAC: Adding query filter params (group.id=f2aa80870e8543bab7b8913e4d9de132, role.id=c4bab4a02e5c4f5f97a024e844a0c072, scope.project.id=4a4ff416931141ce83908e05cc434590) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:04:39.705617 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7beb5ba-42b8-42e4-be1d-640127ee5fab None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:39.707138 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7beb5ba-42b8-42e4-be1d-640127ee5fab None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:39.713070 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 214/439] 10.222.0.26 () {56 vars in 1376 bytes} [Thu Jul 14 09:04:39 2022] GET /identity/v3/role_assignments?group.id=f2aa80870e8543bab7b8913e4d9de132&scope.project.id=4a4ff416931141ce83908e05cc434590&role.id=c4bab4a02e5c4f5f97a024e844a0c072 => generated 611 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:04:39.716059 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7beb5ba-42b8-42e4-be1d-640127ee5fab None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:39.734626 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d9396b-70f1-4a2a-8855-92e3821f3606 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:39.785697 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7964e498-57db-4644-8e21-6c5c0133f493 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:39.786514 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7964e498-57db-4644-8e21-6c5c0133f493 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:39.786796 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7964e498-57db-4644-8e21-6c5c0133f493 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:39.787072 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7964e498-57db-4644-8e21-6c5c0133f493 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:39.799904 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7964e498-57db-4644-8e21-6c5c0133f493 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 09:04:39.808148 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-7964e498-57db-4644-8e21-6c5c0133f493 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 09:04:39.808148 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:04:39.808148 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:04:39.808148 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:04:39.808148 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:04:39.808148 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:04:39.808148 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:04:39.808148 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:04:39.808148 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:04:39.808148 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:04:39.808148 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:04:39.808148 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:04:39.808148 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 09:04:39.808148 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 09:04:39.808148 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 09:04:39.808148 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 09:04:39.809150 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 09:04:39.809150 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 09:04:39.809150 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 09:04:39.809150 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 09:04:39.809150 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:04:39.809150 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:04:39.809150 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:04:39.809150 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:04:39.809150 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:04:39.809150 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:04:39.809150 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:04:39.809150 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 14 09:04:39.809150 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:04:39.809150 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:04:39.809150 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:04:39.809150 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:04:39.809969 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:04:39.809969 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:04:39.809969 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:04:39.809969 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:04:39.809969 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 09:04:39.809969 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 09:04:39.809969 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 09:04:39.809969 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 09:04:39.809969 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 09:04:39.809969 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 09:04:39.809969 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 09:04:39.809969 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 14 09:04:39.814505 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 226/440] 10.222.0.26 () {56 vars in 1114 bytes} [Thu Jul 14 09:04:39 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:04:39.815480 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7964e498-57db-4644-8e21-6c5c0133f493 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:39.842853 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:39.849250 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7beb5ba-42b8-42e4-be1d-640127ee5fab None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:39.889605 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fc8ab3-e0b6-42be-b665-41539b31d145 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:39.890846 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fc8ab3-e0b6-42be-b665-41539b31d145 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:39.892383 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fc8ab3-e0b6-42be-b665-41539b31d145 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:39.892383 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fc8ab3-e0b6-42be-b665-41539b31d145 None admin] PATH_INFO: `/v3/groups/f2aa80870e8543bab7b8913e4d9de132` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:39.892722 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23fc8ab3-e0b6-42be-b665-41539b31d145 None admin] RBAC: Authorizing `identity:get_group(group_id=f2aa80870e8543bab7b8913e4d9de132)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:39.894631 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23fc8ab3-e0b6-42be-b665-41539b31d145 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:39.896238 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 215/441] 10.222.0.26 () {56 vars in 1180 bytes} [Thu Jul 14 09:04:39 2022] GET /identity/v3/groups/f2aa80870e8543bab7b8913e4d9de132 => generated 226 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:04:39.902616 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fc8ab3-e0b6-42be-b665-41539b31d145 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:39.917430 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:39.938771 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7964e498-57db-4644-8e21-6c5c0133f493 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:39.981155 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6a5e54-ca14-4c9d-a8b1-aa115fbd7df1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:39.981928 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6a5e54-ca14-4c9d-a8b1-aa115fbd7df1 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:39.982214 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6a5e54-ca14-4c9d-a8b1-aa115fbd7df1 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:39.982489 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6a5e54-ca14-4c9d-a8b1-aa115fbd7df1 None admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:39.984254 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6a5e54-ca14-4c9d-a8b1-aa115fbd7df1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:04:39.984254 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6a5e54-ca14-4c9d-a8b1-aa115fbd7df1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:39.987239 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6a5e54-ca14-4c9d-a8b1-aa115fbd7df1 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:39.996671 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 227/442] 10.222.0.26 () {56 vars in 1111 bytes} [Thu Jul 14 09:04:39 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:04:40.003521 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6a5e54-ca14-4c9d-a8b1-aa115fbd7df1 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:40.024687 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fc8ab3-e0b6-42be-b665-41539b31d145 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:40.074490 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57661f4c-62d9-4398-8f40-cff7b0f2f485 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:40.075118 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57661f4c-62d9-4398-8f40-cff7b0f2f485 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:40.075458 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57661f4c-62d9-4398-8f40-cff7b0f2f485 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:40.075736 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57661f4c-62d9-4398-8f40-cff7b0f2f485 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:40.076609 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57661f4c-62d9-4398-8f40-cff7b0f2f485 None admin] RBAC: Adding query filter params (group.id=f2aa80870e8543bab7b8913e4d9de132, role.id=d1691322ce6a4edab7ce722edf0f2988, scope.project.id=a596189576264bfc9638ca33f184759c) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:04:40.076924 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57661f4c-62d9-4398-8f40-cff7b0f2f485 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:40.078362 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57661f4c-62d9-4398-8f40-cff7b0f2f485 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:40.087228 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 216/443] 10.222.0.26 () {56 vars in 1376 bytes} [Thu Jul 14 09:04:39 2022] GET /identity/v3/role_assignments?group.id=f2aa80870e8543bab7b8913e4d9de132&scope.project.id=a596189576264bfc9638ca33f184759c&role.id=d1691322ce6a4edab7ce722edf0f2988 => generated 611 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 09:04:40.124481 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:40.144976 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6a5e54-ca14-4c9d-a8b1-aa115fbd7df1 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:40.183921 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc31771-e212-4f2b-a6b7-e387c9f53e1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:40.184956 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc31771-e212-4f2b-a6b7-e387c9f53e1a None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:40.185435 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc31771-e212-4f2b-a6b7-e387c9f53e1a None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:40.185902 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc31771-e212-4f2b-a6b7-e387c9f53e1a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:40.261323 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc31771-e212-4f2b-a6b7-e387c9f53e1a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:40.262819 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc31771-e212-4f2b-a6b7-e387c9f53e1a None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:40.353806 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:40.371063 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 228/444] 10.222.0.26 () {58 vars in 1282 bytes} [Thu Jul 14 09:04:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 14 09:04:40.376880 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57661f4c-62d9-4398-8f40-cff7b0f2f485 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:40.429481 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8e9df4-c9f4-448c-af1c-478b80f52e18 None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:40.429763 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8e9df4-c9f4-448c-af1c-478b80f52e18 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:40.430033 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8e9df4-c9f4-448c-af1c-478b80f52e18 None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:40.431719 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 229/445] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:04:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 09:04:40.436694 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262a36d6-6a41-484a-854c-a495c4c9d770 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:40.436974 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262a36d6-6a41-484a-854c-a495c4c9d770 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:40.437251 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262a36d6-6a41-484a-854c-a495c4c9d770 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:40.455047 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-262a36d6-6a41-484a-854c-a495c4c9d770 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:04:40.455987 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:40.456252 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-262a36d6-6a41-484a-854c-a495c4c9d770 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:04:40.484885 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57661f4c-62d9-4398-8f40-cff7b0f2f485 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:40.518335 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3d484b-acf0-4685-943d-02f0c9f94fd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:40.519210 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3d484b-acf0-4685-943d-02f0c9f94fd2 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:40.519502 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3d484b-acf0-4685-943d-02f0c9f94fd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:40.519760 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3d484b-acf0-4685-943d-02f0c9f94fd2 None admin] PATH_INFO: `/v3/projects/a596189576264bfc9638ca33f184759c` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:40.520942 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3d484b-acf0-4685-943d-02f0c9f94fd2 None admin] RBAC: Authorizing `identity:get_project(project_id=a596189576264bfc9638ca33f184759c)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:40.522443 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3d484b-acf0-4685-943d-02f0c9f94fd2 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:40.524262 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 217/446] 10.222.0.26 () {56 vars in 1186 bytes} [Thu Jul 14 09:04:40 2022] GET /identity/v3/projects/a596189576264bfc9638ca33f184759c => generated 302 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 09:04:40.524365 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 230/447] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:04:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 09:04:40.529501 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3d484b-acf0-4685-943d-02f0c9f94fd2 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:40.547327 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262a36d6-6a41-484a-854c-a495c4c9d770 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:40.627679 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3d484b-acf0-4685-943d-02f0c9f94fd2 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:40.642757 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262a36d6-6a41-484a-854c-a495c4c9d770 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:40.656359 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:40.661628 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b73cc20-d18a-4c7d-bf9b-3018986eb211 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:40.662502 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b73cc20-d18a-4c7d-bf9b-3018986eb211 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:40.662781 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b73cc20-d18a-4c7d-bf9b-3018986eb211 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:40.663048 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b73cc20-d18a-4c7d-bf9b-3018986eb211 None admin] PATH_INFO: `/v3/groups/f2aa80870e8543bab7b8913e4d9de132` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:40.664246 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b73cc20-d18a-4c7d-bf9b-3018986eb211 None admin] RBAC: Authorizing `identity:get_group(group_id=f2aa80870e8543bab7b8913e4d9de132)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:40.665704 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b73cc20-d18a-4c7d-bf9b-3018986eb211 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:40.667364 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 218/448] 10.222.0.26 () {56 vars in 1180 bytes} [Thu Jul 14 09:04:40 2022] GET /identity/v3/groups/f2aa80870e8543bab7b8913e4d9de132 => generated 226 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:04:40.673583 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b73cc20-d18a-4c7d-bf9b-3018986eb211 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:40.677598 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0eb93b-8b2f-42fc-859e-2f60b26e576a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:40.678390 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0eb93b-8b2f-42fc-859e-2f60b26e576a None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:40.678669 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0eb93b-8b2f-42fc-859e-2f60b26e576a None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:40.678936 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0eb93b-8b2f-42fc-859e-2f60b26e576a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:40.685825 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:40.689408 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0eb93b-8b2f-42fc-859e-2f60b26e576a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 09:04:40.696982 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-4c0eb93b-8b2f-42fc-859e-2f60b26e576a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 09:04:40.696982 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:04:40.696982 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:04:40.696982 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:04:40.696982 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:04:40.696982 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:04:40.696982 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:04:40.696982 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:04:40.696982 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:04:40.696982 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:04:40.696982 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:04:40.696982 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:04:40.696982 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 09:04:40.696982 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 09:04:40.696982 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 09:04:40.696982 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 09:04:40.697862 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 09:04:40.697862 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 09:04:40.697862 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 09:04:40.697862 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 09:04:40.697862 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:04:40.697862 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:04:40.697862 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:04:40.697862 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:04:40.697862 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:04:40.697862 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:04:40.697862 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:04:40.697862 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 14 09:04:40.697862 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:04:40.697862 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:04:40.697862 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:04:40.697862 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:04:40.698862 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:04:40.698862 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:04:40.698862 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:04:40.698862 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:04:40.698862 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 09:04:40.698862 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 09:04:40.698862 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 09:04:40.698862 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 09:04:40.698862 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 09:04:40.698862 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 09:04:40.698862 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 09:04:40.698862 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 14 09:04:40.698862 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 231/449] 10.222.0.26 () {56 vars in 1096 bytes} [Thu Jul 14 09:04:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 09:04:40.703904 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0eb93b-8b2f-42fc-859e-2f60b26e576a None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:40.773209 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b73cc20-d18a-4c7d-bf9b-3018986eb211 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:40.801340 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0eb93b-8b2f-42fc-859e-2f60b26e576a None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:40.806486 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac182acc-6a03-4f61-81b2-ad8d9dadb32f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:40.807136 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac182acc-6a03-4f61-81b2-ad8d9dadb32f None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:40.807450 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac182acc-6a03-4f61-81b2-ad8d9dadb32f None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:40.807723 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac182acc-6a03-4f61-81b2-ad8d9dadb32f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:40.808552 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac182acc-6a03-4f61-81b2-ad8d9dadb32f None admin] RBAC: Adding query filter params (group.id=f2aa80870e8543bab7b8913e4d9de132, role.id=c4bab4a02e5c4f5f97a024e844a0c072, scope.project.id=a596189576264bfc9638ca33f184759c) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:04:40.808862 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac182acc-6a03-4f61-81b2-ad8d9dadb32f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:40.810313 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac182acc-6a03-4f61-81b2-ad8d9dadb32f None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:40.816101 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 219/450] 10.222.0.26 () {56 vars in 1376 bytes} [Thu Jul 14 09:04:40 2022] GET /identity/v3/role_assignments?group.id=f2aa80870e8543bab7b8913e4d9de132&scope.project.id=a596189576264bfc9638ca33f184759c&role.id=c4bab4a02e5c4f5f97a024e844a0c072 => generated 611 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 09:04:40.835434 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e8dc3c-d62f-48cd-b334-09de5c06af2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:40.836136 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e8dc3c-d62f-48cd-b334-09de5c06af2a None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:40.836416 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e8dc3c-d62f-48cd-b334-09de5c06af2a None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:40.836693 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e8dc3c-d62f-48cd-b334-09de5c06af2a None admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:40.837473 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e8dc3c-d62f-48cd-b334-09de5c06af2a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:04:40.837779 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e8dc3c-d62f-48cd-b334-09de5c06af2a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:40.839218 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e8dc3c-d62f-48cd-b334-09de5c06af2a None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:40.848784 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 232/451] 10.222.0.26 () {56 vars in 1099 bytes} [Thu Jul 14 09:04:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:04:40.854694 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac182acc-6a03-4f61-81b2-ad8d9dadb32f None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:40.943232 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:40.971869 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac182acc-6a03-4f61-81b2-ad8d9dadb32f None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:41.007534 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6503bf42-549a-4bd4-bd79-44fed6c398fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:41.008188 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6503bf42-549a-4bd4-bd79-44fed6c398fc None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:41.008474 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6503bf42-549a-4bd4-bd79-44fed6c398fc None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:41.008750 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6503bf42-549a-4bd4-bd79-44fed6c398fc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:41.079247 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6503bf42-549a-4bd4-bd79-44fed6c398fc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:41.080706 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6503bf42-549a-4bd4-bd79-44fed6c398fc None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:41.155877 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:41.183384 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 220/452] 10.222.0.26 () {58 vars in 1282 bytes} [Thu Jul 14 09:04:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 09:04:41.189253 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e8dc3c-d62f-48cd-b334-09de5c06af2a None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:41.224535 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:41.286204 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e8dc3c-d62f-48cd-b334-09de5c06af2a None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:41.323979 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51a1afa-c0c1-4dbd-80b8-d0db678d1564 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:41.324826 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51a1afa-c0c1-4dbd-80b8-d0db678d1564 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:41.325102 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51a1afa-c0c1-4dbd-80b8-d0db678d1564 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:41.325374 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51a1afa-c0c1-4dbd-80b8-d0db678d1564 None admin] PATH_INFO: `/v3/projects/2fd144ddf49f41d58d6674bed415082b` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:41.326615 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51a1afa-c0c1-4dbd-80b8-d0db678d1564 None admin] RBAC: Authorizing `identity:get_project(project_id=2fd144ddf49f41d58d6674bed415082b)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:41.328085 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51a1afa-c0c1-4dbd-80b8-d0db678d1564 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:41.329805 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 233/453] 10.222.0.26 () {56 vars in 1186 bytes} [Thu Jul 14 09:04:41 2022] GET /identity/v3/projects/2fd144ddf49f41d58d6674bed415082b => generated 344 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:04:41.336033 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6503bf42-549a-4bd4-bd79-44fed6c398fc None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:41.441213 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6503bf42-549a-4bd4-bd79-44fed6c398fc None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:41.476827 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3631c7fa-ea50-4f9d-a787-e28bcbb38ef2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:41.477603 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3631c7fa-ea50-4f9d-a787-e28bcbb38ef2 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:41.477891 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3631c7fa-ea50-4f9d-a787-e28bcbb38ef2 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:41.478166 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3631c7fa-ea50-4f9d-a787-e28bcbb38ef2 None admin] PATH_INFO: `/v3/groups/ae3d183691c04d43b40da20c4381678e` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:41.479413 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3631c7fa-ea50-4f9d-a787-e28bcbb38ef2 None admin] RBAC: Authorizing `identity:get_group(group_id=ae3d183691c04d43b40da20c4381678e)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:41.480846 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3631c7fa-ea50-4f9d-a787-e28bcbb38ef2 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:41.482448 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 221/454] 10.222.0.26 () {56 vars in 1180 bytes} [Thu Jul 14 09:04:41 2022] GET /identity/v3/groups/ae3d183691c04d43b40da20c4381678e => generated 229 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:04:41.489611 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51a1afa-c0c1-4dbd-80b8-d0db678d1564 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:41.605345 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51a1afa-c0c1-4dbd-80b8-d0db678d1564 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:41.607688 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:41.644416 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de7de6f-bcfc-4307-b3d2-c027d177acdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:41.645068 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de7de6f-bcfc-4307-b3d2-c027d177acdc None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:41.645348 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de7de6f-bcfc-4307-b3d2-c027d177acdc None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:41.645626 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de7de6f-bcfc-4307-b3d2-c027d177acdc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:41.646468 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de7de6f-bcfc-4307-b3d2-c027d177acdc None admin] RBAC: Adding query filter params (group.id=ae3d183691c04d43b40da20c4381678e, role.id=7b549d58d1724395b343609d895c80d0, scope.project.id=2fd144ddf49f41d58d6674bed415082b) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:04:41.646779 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de7de6f-bcfc-4307-b3d2-c027d177acdc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:41.648279 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de7de6f-bcfc-4307-b3d2-c027d177acdc None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:41.654302 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 234/455] 10.222.0.26 () {56 vars in 1376 bytes} [Thu Jul 14 09:04:41 2022] GET /identity/v3/role_assignments?group.id=ae3d183691c04d43b40da20c4381678e&scope.project.id=2fd144ddf49f41d58d6674bed415082b&role.id=7b549d58d1724395b343609d895c80d0 => generated 611 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:04:44.031485 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf153f8-d360-4b8f-a66f-ecad21878ca7 None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:44.032498 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf153f8-d360-4b8f-a66f-ecad21878ca7 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:44.032986 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf153f8-d360-4b8f-a66f-ecad21878ca7 None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:44.034574 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 222/456] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:04:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:04:44.039916 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbc380c-c462-47b9-8f88-19d56d0efeda None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:44.040316 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbc380c-c462-47b9-8f88-19d56d0efeda None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:44.040489 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbc380c-c462-47b9-8f88-19d56d0efeda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:44.060874 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-fdbc380c-c462-47b9-8f88-19d56d0efeda None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:04:44.062128 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-fdbc380c-c462-47b9-8f88-19d56d0efeda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:04:44.152170 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93aac8ca-a1e0-488f-ae61-2d87499a4105 None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:44.152517 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93aac8ca-a1e0-488f-ae61-2d87499a4105 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:44.152746 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93aac8ca-a1e0-488f-ae61-2d87499a4105 None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:44.152847 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 235/457] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:04:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 09:04:44.154262 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 223/458] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:04:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:04:44.176882 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce44cf4-d04d-4a6f-a2dd-57371cd39469 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:44.177218 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce44cf4-d04d-4a6f-a2dd-57371cd39469 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:44.177489 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce44cf4-d04d-4a6f-a2dd-57371cd39469 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:44.182227 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93aac8ca-a1e0-488f-ae61-2d87499a4105 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:44.200472 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-7ce44cf4-d04d-4a6f-a2dd-57371cd39469 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:04:44.201661 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-7ce44cf4-d04d-4a6f-a2dd-57371cd39469 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:04:44.279585 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:44.340775 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 236/459] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:04:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 09:04:44.368818 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93aac8ca-a1e0-488f-ae61-2d87499a4105 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:44.372876 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce44cf4-d04d-4a6f-a2dd-57371cd39469 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:44.403662 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:44.437867 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b025fbb-cbd6-471e-af5a-c0ca491ae0e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:44.438678 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b025fbb-cbd6-471e-af5a-c0ca491ae0e5 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:44.438960 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b025fbb-cbd6-471e-af5a-c0ca491ae0e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:44.439254 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b025fbb-cbd6-471e-af5a-c0ca491ae0e5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:44.461860 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b025fbb-cbd6-471e-af5a-c0ca491ae0e5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:44.463370 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b025fbb-cbd6-471e-af5a-c0ca491ae0e5 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:44.495011 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-8b025fbb-cbd6-471e-af5a-c0ca491ae0e5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 09:04:44.495011 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:04:44.495011 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:04:44.495011 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:04:44.495011 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:04:44.495011 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:04:44.495011 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:04:44.495011 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:04:44.495011 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:04:44.495011 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:04:44.495011 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:04:44.495011 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:04:44.495011 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 09:04:44.495011 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 09:04:44.495011 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 09:04:44.495998 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 09:04:44.495998 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:04:44.495998 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:04:44.495998 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:04:44.495998 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:04:44.495998 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:04:44.495998 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:04:44.495998 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:04:44.495998 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 14 09:04:44.495998 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:04:44.495998 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:04:44.495998 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:04:44.495998 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:04:44.495998 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:04:44.495998 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:04:44.495998 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:04:44.495998 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:04:44.497284 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 09:04:44.497284 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 09:04:44.497284 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 09:04:44.497284 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 14 09:04:44.497284 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 224/460] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jul 14 09:04:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 314 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:04:44.504126 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b025fbb-cbd6-471e-af5a-c0ca491ae0e5 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:44.524179 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce44cf4-d04d-4a6f-a2dd-57371cd39469 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:44.593833 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e68a602-82ff-47f1-82b1-4dd3d6dd08e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:44.594563 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e68a602-82ff-47f1-82b1-4dd3d6dd08e1 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:44.594843 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e68a602-82ff-47f1-82b1-4dd3d6dd08e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:44.595200 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e68a602-82ff-47f1-82b1-4dd3d6dd08e1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:44.611901 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e68a602-82ff-47f1-82b1-4dd3d6dd08e1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:44.613397 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e68a602-82ff-47f1-82b1-4dd3d6dd08e1 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:44.648027 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-6e68a602-82ff-47f1-82b1-4dd3d6dd08e1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 09:04:44.648027 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:04:44.648027 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:04:44.648027 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:04:44.648027 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:04:44.648027 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:04:44.648027 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:04:44.648027 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:04:44.648027 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:04:44.648027 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:04:44.648027 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:04:44.648027 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:04:44.648027 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 09:04:44.648027 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 09:04:44.648027 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 09:04:44.648958 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 09:04:44.648958 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:04:44.648958 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:04:44.648958 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:04:44.648958 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:04:44.648958 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:04:44.648958 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:04:44.648958 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:04:44.648958 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 14 09:04:44.648958 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:04:44.648958 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:04:44.648958 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:04:44.648958 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:04:44.648958 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:04:44.648958 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:04:44.648958 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:04:44.648958 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:04:44.649813 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 09:04:44.649813 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 09:04:44.649813 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 09:04:44.649813 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 14 09:04:44.655276 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 237/461] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jul 14 09:04:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:04:44.658538 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e68a602-82ff-47f1-82b1-4dd3d6dd08e1 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:44.687560 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:44.691740 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b025fbb-cbd6-471e-af5a-c0ca491ae0e5 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:44.730197 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2464db-9a24-41a5-9fa0-6580b99a6289 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:44.730865 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2464db-9a24-41a5-9fa0-6580b99a6289 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:44.731146 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2464db-9a24-41a5-9fa0-6580b99a6289 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:44.731449 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2464db-9a24-41a5-9fa0-6580b99a6289 None admin] PATH_INFO: `/v3/domains` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:44.732279 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2464db-9a24-41a5-9fa0-6580b99a6289 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:04:44.732585 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2464db-9a24-41a5-9fa0-6580b99a6289 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:44.734145 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2464db-9a24-41a5-9fa0-6580b99a6289 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:44.746970 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 225/462] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Jul 14 09:04:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:04:44.748271 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1b5601-25d0-44ea-b0a6-ce87307ad013 None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:44.748557 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1b5601-25d0-44ea-b0a6-ce87307ad013 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:44.748834 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1b5601-25d0-44ea-b0a6-ce87307ad013 None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:44.750324 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 226/463] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:04:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:04:44.752785 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f0610b-7686-4d4f-96a5-551db63c7431 None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:44.753066 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f0610b-7686-4d4f-96a5-551db63c7431 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:44.753338 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f0610b-7686-4d4f-96a5-551db63c7431 None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:44.755772 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 227/464] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:04:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:04:44.756435 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f0610b-7686-4d4f-96a5-551db63c7431 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:44.784396 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:44.805444 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e68a602-82ff-47f1-82b1-4dd3d6dd08e1 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:44.840132 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c049af-db70-42e8-8928-92524a44ca9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:44.840793 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c049af-db70-42e8-8928-92524a44ca9b None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:44.841098 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c049af-db70-42e8-8928-92524a44ca9b None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:44.841596 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c049af-db70-42e8-8928-92524a44ca9b None admin] PATH_INFO: `/v3/domains` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:44.842069 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c049af-db70-42e8-8928-92524a44ca9b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:04:44.842444 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c049af-db70-42e8-8928-92524a44ca9b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:44.843831 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c049af-db70-42e8-8928-92524a44ca9b None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:44.856344 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 238/465] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Jul 14 09:04:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:04:44.857497 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276ddcac-1a3b-4fc2-b93c-872a71438557 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:44.857784 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276ddcac-1a3b-4fc2-b93c-872a71438557 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:44.858057 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276ddcac-1a3b-4fc2-b93c-872a71438557 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:44.865247 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f0610b-7686-4d4f-96a5-551db63c7431 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:44.874272 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-276ddcac-1a3b-4fc2-b93c-872a71438557 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:04:44.875480 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-276ddcac-1a3b-4fc2-b93c-872a71438557 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:04:44.904008 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500becbe-ba8e-43e4-8649-e7964dd65b02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:44.904748 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500becbe-ba8e-43e4-8649-e7964dd65b02 None admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:44.905028 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500becbe-ba8e-43e4-8649-e7964dd65b02 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:44.905302 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500becbe-ba8e-43e4-8649-e7964dd65b02 None admin] PATH_INFO: `/v3/users` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:44.906134 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500becbe-ba8e-43e4-8649-e7964dd65b02 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:44.908152 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500becbe-ba8e-43e4-8649-e7964dd65b02 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:44.939236 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 228/466] 10.222.0.26 () {60 vars in 1132 bytes} [Thu Jul 14 09:04:44 2022] POST /identity/v3/users => generated 246 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 09:04:44.939236 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725303e0-4bde-464a-b9ca-d949a6367414 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:44.939236 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725303e0-4bde-464a-b9ca-d949a6367414 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:44.939236 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725303e0-4bde-464a-b9ca-d949a6367414 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:44.952491 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-725303e0-4bde-464a-b9ca-d949a6367414 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:04:44.953711 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-725303e0-4bde-464a-b9ca-d949a6367414 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:04:44.957886 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:44.974166 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 239/467] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:04:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 09:04:44.978803 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276ddcac-1a3b-4fc2-b93c-872a71438557 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:45.051604 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 229/468] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:04:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 09:04:45.052648 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725303e0-4bde-464a-b9ca-d949a6367414 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:45.063413 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:45.095746 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276ddcac-1a3b-4fc2-b93c-872a71438557 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:45.133991 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa22948c-e100-4b5b-b842-8e5f19548d8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:45.134731 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa22948c-e100-4b5b-b842-8e5f19548d8b None admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:45.135146 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa22948c-e100-4b5b-b842-8e5f19548d8b None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:45.135464 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa22948c-e100-4b5b-b842-8e5f19548d8b None admin] PATH_INFO: `/v3/users` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:45.136413 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa22948c-e100-4b5b-b842-8e5f19548d8b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:45.138667 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa22948c-e100-4b5b-b842-8e5f19548d8b None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:45.163400 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 240/469] 10.222.0.26 () {60 vars in 1132 bytes} [Thu Jul 14 09:04:44 2022] POST /identity/v3/users => generated 248 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 09:04:45.164056 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa22948c-e100-4b5b-b842-8e5f19548d8b None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:45.171234 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725303e0-4bde-464a-b9ca-d949a6367414 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:45.216564 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecf7327-6aa1-4bdc-bc0d-37f931687121 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:45.217508 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecf7327-6aa1-4bdc-bc0d-37f931687121 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:45.217713 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecf7327-6aa1-4bdc-bc0d-37f931687121 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:45.218064 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecf7327-6aa1-4bdc-bc0d-37f931687121 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:45.231081 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:45.234949 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecf7327-6aa1-4bdc-bc0d-37f931687121 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:45.236563 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecf7327-6aa1-4bdc-bc0d-37f931687121 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:45.262016 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-6ecf7327-6aa1-4bdc-bc0d-37f931687121 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 09:04:45.262016 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:04:45.262016 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:04:45.262016 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:04:45.262016 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:04:45.262016 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:04:45.262016 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:04:45.262016 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:04:45.262016 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:04:45.262016 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:04:45.262016 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:04:45.262016 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:04:45.262016 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 09:04:45.262016 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 09:04:45.262016 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 09:04:45.262928 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 09:04:45.262928 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:04:45.262928 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:04:45.262928 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:04:45.262928 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:04:45.262928 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:04:45.262928 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:04:45.262928 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:04:45.262928 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 14 09:04:45.262928 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:04:45.262928 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:04:45.262928 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:04:45.262928 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:04:45.262928 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:04:45.262928 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:04:45.262928 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:04:45.262928 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:04:45.263849 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 09:04:45.263849 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 09:04:45.263849 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 09:04:45.263849 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 14 09:04:45.263849 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 230/470] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jul 14 09:04:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:04:45.271014 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecf7327-6aa1-4bdc-bc0d-37f931687121 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:45.326657 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa22948c-e100-4b5b-b842-8e5f19548d8b None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:45.364334 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:45.371389 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50ffa89-e9c3-4f1d-8917-6845af6f4b7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:45.372757 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50ffa89-e9c3-4f1d-8917-6845af6f4b7b None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:45.373111 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50ffa89-e9c3-4f1d-8917-6845af6f4b7b None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:45.373459 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50ffa89-e9c3-4f1d-8917-6845af6f4b7b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:45.388327 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50ffa89-e9c3-4f1d-8917-6845af6f4b7b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:45.389877 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50ffa89-e9c3-4f1d-8917-6845af6f4b7b None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:45.410225 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecf7327-6aa1-4bdc-bc0d-37f931687121 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:45.416186 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-a50ffa89-e9c3-4f1d-8917-6845af6f4b7b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 09:04:45.416186 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:04:45.416186 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:04:45.416186 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:04:45.416186 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:04:45.416186 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:04:45.416186 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:04:45.416186 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:04:45.416186 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:04:45.416186 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:04:45.416186 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:04:45.416186 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:04:45.416186 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 09:04:45.416186 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 09:04:45.416186 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 09:04:45.417025 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 09:04:45.417025 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:04:45.417025 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:04:45.417025 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:04:45.417025 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:04:45.417025 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:04:45.417025 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:04:45.417025 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:04:45.417025 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 14 09:04:45.417025 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:04:45.417025 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:04:45.417025 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:04:45.417025 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:04:45.417025 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:04:45.417025 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:04:45.417025 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:04:45.417025 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:04:45.417862 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 09:04:45.417862 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 09:04:45.417862 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 09:04:45.417862 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 14 09:04:45.417862 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 241/471] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jul 14 09:04:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:04:45.423794 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50ffa89-e9c3-4f1d-8917-6845af6f4b7b None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:45.451974 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679b13e7-82d2-4d47-9fcf-1840cc2f707b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:45.452892 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679b13e7-82d2-4d47-9fcf-1840cc2f707b None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:45.453473 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679b13e7-82d2-4d47-9fcf-1840cc2f707b None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:45.453766 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679b13e7-82d2-4d47-9fcf-1840cc2f707b None admin] PATH_INFO: `/v3/domains` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:45.454777 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679b13e7-82d2-4d47-9fcf-1840cc2f707b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:04:45.455087 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679b13e7-82d2-4d47-9fcf-1840cc2f707b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:45.457032 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679b13e7-82d2-4d47-9fcf-1840cc2f707b None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:45.471727 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 231/472] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Jul 14 09:04:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 09:04:45.477107 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679b13e7-82d2-4d47-9fcf-1840cc2f707b None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:45.537713 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50ffa89-e9c3-4f1d-8917-6845af6f4b7b None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:45.549420 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:45.585219 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cf4fae-b773-4750-b539-c2e60047951a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:45.586661 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cf4fae-b773-4750-b539-c2e60047951a None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:45.587161 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cf4fae-b773-4750-b539-c2e60047951a None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:45.587683 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cf4fae-b773-4750-b539-c2e60047951a None admin] PATH_INFO: `/v3/domains` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:45.588926 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cf4fae-b773-4750-b539-c2e60047951a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:04:45.589727 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cf4fae-b773-4750-b539-c2e60047951a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:45.591647 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cf4fae-b773-4750-b539-c2e60047951a None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:45.607820 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 242/473] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Jul 14 09:04:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 09:04:45.609606 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579e62ec-e076-448d-8e56-5e98d0536f3a None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:45.610086 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579e62ec-e076-448d-8e56-5e98d0536f3a None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:45.610549 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579e62ec-e076-448d-8e56-5e98d0536f3a None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:45.612008 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 243/474] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:04:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:04:45.614348 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579e62ec-e076-448d-8e56-5e98d0536f3a None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:45.616042 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679b13e7-82d2-4d47-9fcf-1840cc2f707b None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:45.654204 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7115ac-7d55-46b8-ba53-abe1e7a3f001 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:45.654702 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7115ac-7d55-46b8-ba53-abe1e7a3f001 None admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:45.654990 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7115ac-7d55-46b8-ba53-abe1e7a3f001 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:45.655355 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7115ac-7d55-46b8-ba53-abe1e7a3f001 None admin] PATH_INFO: `/v3/users` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:45.656275 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7115ac-7d55-46b8-ba53-abe1e7a3f001 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:45.658010 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7115ac-7d55-46b8-ba53-abe1e7a3f001 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:45.666929 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:45.684102 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 232/475] 10.222.0.26 () {60 vars in 1132 bytes} [Thu Jul 14 09:04:45 2022] POST /identity/v3/users => generated 249 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 09:04:45.685897 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7741827-1109-49d6-9184-1db83b0acfd1 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:45.686570 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7741827-1109-49d6-9184-1db83b0acfd1 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:45.686766 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7741827-1109-49d6-9184-1db83b0acfd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:45.707625 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-e7741827-1109-49d6-9184-1db83b0acfd1 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:04:45.708640 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-e7741827-1109-49d6-9184-1db83b0acfd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:04:45.734843 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579e62ec-e076-448d-8e56-5e98d0536f3a None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:45.770823 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb096007-c88f-4a82-9c07-38dfed640e5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:45.771964 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb096007-c88f-4a82-9c07-38dfed640e5a None admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:45.772446 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb096007-c88f-4a82-9c07-38dfed640e5a None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:45.772913 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb096007-c88f-4a82-9c07-38dfed640e5a None admin] PATH_INFO: `/v3/users` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:45.773843 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb096007-c88f-4a82-9c07-38dfed640e5a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:45.775701 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb096007-c88f-4a82-9c07-38dfed640e5a None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:45.789523 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 233/476] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:04:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 09:04:45.803593 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 244/477] 10.222.0.26 () {60 vars in 1132 bytes} [Thu Jul 14 09:04:45 2022] POST /identity/v3/users => generated 248 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 09:04:45.833522 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7741827-1109-49d6-9184-1db83b0acfd1 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:45.899020 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:45.977740 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7741827-1109-49d6-9184-1db83b0acfd1 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:46.016905 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6e8734-d3e4-4272-a139-80d4a8b04cc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:46.017587 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6e8734-d3e4-4272-a139-80d4a8b04cc4 None admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:46.017870 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6e8734-d3e4-4272-a139-80d4a8b04cc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:46.018148 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6e8734-d3e4-4272-a139-80d4a8b04cc4 None admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:46.018898 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6e8734-d3e4-4272-a139-80d4a8b04cc4 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:46.020305 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6e8734-d3e4-4272-a139-80d4a8b04cc4 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:46.027594 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.sql.core [None req-6e6e8734-d3e4-4272-a139-80d4a8b04cc4 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 14 09:04:46.027594 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 14 09:04:46.027594 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [parameters: {'id': '00dee44cc8d144929032a238fbf4d929', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 14 09:04:46.027594 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80381) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 14 09:04:46.044594 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-6e6e8734-d3e4-4272-a139-80d4a8b04cc4 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 14 09:04:46.044594 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:04:46.044594 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 14 09:04:46.044594 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 14 09:04:46.044594 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 14 09:04:46.044594 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 14 09:04:46.044594 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 14 09:04:46.044594 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application result = self._query(query) Jul 14 09:04:46.044594 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 14 09:04:46.044594 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application conn.query(q) Jul 14 09:04:46.044594 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 14 09:04:46.044594 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 14 09:04:46.044594 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 14 09:04:46.044594 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application result.read() Jul 14 09:04:46.044594 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 14 09:04:46.045486 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 14 09:04:46.045486 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 14 09:04:46.045486 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 14 09:04:46.045486 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 14 09:04:46.045486 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 14 09:04:46.045486 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 14 09:04:46.045486 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 14 09:04:46.045486 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 14 09:04:46.045486 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 14 09:04:46.045486 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 14 09:04:46.045486 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 14 09:04:46.045486 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:04:46.045486 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 14 09:04:46.045486 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 14 09:04:46.045486 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jul 14 09:04:46.045486 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return ref.to_dict() Jul 14 09:04:46.045486 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 14 09:04:46.045486 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application next(self.gen) Jul 14 09:04:46.046540 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 14 09:04:46.046540 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application yield resource Jul 14 09:04:46.046540 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 14 09:04:46.046540 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application next(self.gen) Jul 14 09:04:46.046540 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 14 09:04:46.046540 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application self.session.rollback() Jul 14 09:04:46.046540 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 14 09:04:46.046540 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application self.force_reraise() Jul 14 09:04:46.046540 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 14 09:04:46.046540 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise self.value Jul 14 09:04:46.046540 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 14 09:04:46.046540 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 14 09:04:46.046540 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 14 09:04:46.046540 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application session.commit() Jul 14 09:04:46.046540 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 14 09:04:46.046540 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 14 09:04:46.046540 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 14 09:04:46.048262 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application self._prepare_impl() Jul 14 09:04:46.048262 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 14 09:04:46.048262 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application self.session.flush() Jul 14 09:04:46.048262 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 14 09:04:46.048262 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application self._flush(objects) Jul 14 09:04:46.048262 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 14 09:04:46.048262 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 14 09:04:46.048262 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 14 09:04:46.048262 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application compat.raise_( Jul 14 09:04:46.048262 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 14 09:04:46.048262 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception Jul 14 09:04:46.048262 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 14 09:04:46.048262 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application flush_context.execute() Jul 14 09:04:46.048262 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 14 09:04:46.048262 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rec.execute(self) Jul 14 09:04:46.048262 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 14 09:04:46.048262 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 14 09:04:46.050264 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 14 09:04:46.050264 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 14 09:04:46.050264 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 14 09:04:46.050264 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 14 09:04:46.050264 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 14 09:04:46.050264 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 14 09:04:46.050264 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 14 09:04:46.050264 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 14 09:04:46.050264 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 14 09:04:46.050264 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 14 09:04:46.050264 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 14 09:04:46.050264 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 14 09:04:46.050264 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 14 09:04:46.050264 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 14 09:04:46.050264 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 14 09:04:46.050264 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception Jul 14 09:04:46.052705 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 14 09:04:46.052705 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 14 09:04:46.052705 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 14 09:04:46.052705 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 14 09:04:46.052705 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 14 09:04:46.052705 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application result = self._query(query) Jul 14 09:04:46.052705 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 14 09:04:46.052705 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application conn.query(q) Jul 14 09:04:46.052705 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 14 09:04:46.052705 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 14 09:04:46.052705 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 14 09:04:46.052705 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application result.read() Jul 14 09:04:46.052705 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 14 09:04:46.052705 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 14 09:04:46.052705 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 14 09:04:46.052705 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 14 09:04:46.054665 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 14 09:04:46.054665 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 14 09:04:46.054665 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 14 09:04:46.054665 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 14 09:04:46.054665 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 14 09:04:46.054665 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 14 09:04:46.054665 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [parameters: {'id': '00dee44cc8d144929032a238fbf4d929', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 14 09:04:46.054665 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 14 09:04:46.054665 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 14 09:04:46.054665 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 09:04:46.054665 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 14 09:04:46.054665 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:04:46.054665 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:04:46.054665 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:04:46.054665 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:04:46.054665 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:04:46.058272 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:04:46.058272 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:04:46.058272 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:04:46.058272 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:04:46.058272 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:04:46.058272 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:04:46.058272 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jul 14 09:04:46.058272 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jul 14 09:04:46.058272 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:04:46.058272 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:04:46.058272 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jul 14 09:04:46.058272 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jul 14 09:04:46.058272 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 14 09:04:46.058272 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 14 09:04:46.058272 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 14 09:04:46.058272 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 14 09:04:46.060212 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 234/478] 10.222.0.26 () {60 vars in 1131 bytes} [Thu Jul 14 09:04:45 2022] POST /identity/v3/roles => generated 144 bytes in 214 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:04:46.060352 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb096007-c88f-4a82-9c07-38dfed640e5a None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:46.153933 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:46.162624 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb096007-c88f-4a82-9c07-38dfed640e5a None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:46.197808 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d47f72-38e6-4747-8e1a-2c405e817d16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:46.198959 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d47f72-38e6-4747-8e1a-2c405e817d16 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:46.199459 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d47f72-38e6-4747-8e1a-2c405e817d16 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:46.199923 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d47f72-38e6-4747-8e1a-2c405e817d16 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:46.213107 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d47f72-38e6-4747-8e1a-2c405e817d16 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 09:04:46.221370 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-c7d47f72-38e6-4747-8e1a-2c405e817d16 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 09:04:46.221370 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:04:46.221370 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:04:46.221370 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:04:46.221370 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:04:46.221370 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:04:46.221370 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:04:46.221370 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:04:46.221370 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:04:46.221370 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:04:46.221370 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:04:46.221370 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:04:46.221370 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 09:04:46.221370 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 09:04:46.221370 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 09:04:46.221370 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 09:04:46.222288 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 09:04:46.222288 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 09:04:46.222288 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 09:04:46.222288 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 09:04:46.222288 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:04:46.222288 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:04:46.222288 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:04:46.222288 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:04:46.222288 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:04:46.222288 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:04:46.222288 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:04:46.222288 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 14 09:04:46.222288 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:04:46.222288 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:04:46.222288 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:04:46.222288 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:04:46.223096 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:04:46.223096 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:04:46.223096 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:04:46.223096 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:04:46.223096 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 09:04:46.223096 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 09:04:46.223096 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 09:04:46.223096 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 09:04:46.223096 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 09:04:46.223096 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 09:04:46.223096 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 09:04:46.223096 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 14 09:04:46.223096 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 245/479] 10.222.0.26 () {56 vars in 1114 bytes} [Thu Jul 14 09:04:46 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 09:04:46.232373 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6e8734-d3e4-4272-a139-80d4a8b04cc4 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:46.350966 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6e8734-d3e4-4272-a139-80d4a8b04cc4 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:46.353118 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:46.394986 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da552e42-23ec-403c-b30c-ca0ae08e688c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:46.395684 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da552e42-23ec-403c-b30c-ca0ae08e688c None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:46.395964 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da552e42-23ec-403c-b30c-ca0ae08e688c None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:46.396237 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da552e42-23ec-403c-b30c-ca0ae08e688c None admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:46.396991 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da552e42-23ec-403c-b30c-ca0ae08e688c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:04:46.397294 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da552e42-23ec-403c-b30c-ca0ae08e688c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:46.398683 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da552e42-23ec-403c-b30c-ca0ae08e688c None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:46.408099 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 235/480] 10.222.0.26 () {56 vars in 1111 bytes} [Thu Jul 14 09:04:46 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:04:47.432377 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ca4584-b78a-4d14-b362-86d24a348f46 None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:47.432942 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ca4584-b78a-4d14-b362-86d24a348f46 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:47.432942 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ca4584-b78a-4d14-b362-86d24a348f46 None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:47.434619 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 246/481] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:04:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:04:47.440267 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7e2101-43fb-42e0-9106-ef14faf20069 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:47.440747 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7e2101-43fb-42e0-9106-ef14faf20069 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:47.440849 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7e2101-43fb-42e0-9106-ef14faf20069 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:47.463954 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-5a7e2101-43fb-42e0-9106-ef14faf20069 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:04:47.465133 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-5a7e2101-43fb-42e0-9106-ef14faf20069 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:04:47.548519 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 236/482] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:04:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 09:04:47.596718 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ca4584-b78a-4d14-b362-86d24a348f46 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:47.736773 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ca4584-b78a-4d14-b362-86d24a348f46 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:47.779116 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406537d3-8b0d-4f13-ba7f-cefb926a4654 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:47.779829 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c5ad89-09e4-455e-941d-50611200d535 None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:47.780024 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406537d3-8b0d-4f13-ba7f-cefb926a4654 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:47.780284 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406537d3-8b0d-4f13-ba7f-cefb926a4654 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:47.780556 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406537d3-8b0d-4f13-ba7f-cefb926a4654 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:47.780922 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c5ad89-09e4-455e-941d-50611200d535 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:47.781394 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c5ad89-09e4-455e-941d-50611200d535 None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:47.786096 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 237/483] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:04:47 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:04:47.790001 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8606a6f-1cd6-43ea-af58-6c5549934cf4 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:47.790501 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8606a6f-1cd6-43ea-af58-6c5549934cf4 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:47.790991 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8606a6f-1cd6-43ea-af58-6c5549934cf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:47.794953 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406537d3-8b0d-4f13-ba7f-cefb926a4654 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 09:04:47.802714 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-406537d3-8b0d-4f13-ba7f-cefb926a4654 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 09:04:47.802714 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:04:47.802714 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:04:47.802714 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:04:47.802714 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:04:47.802714 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:04:47.802714 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:04:47.802714 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:04:47.802714 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:04:47.802714 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:04:47.802714 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:04:47.802714 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:04:47.802714 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 09:04:47.802714 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 09:04:47.802714 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 09:04:47.802714 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 09:04:47.803940 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 09:04:47.803940 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 09:04:47.803940 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 09:04:47.803940 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 09:04:47.803940 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:04:47.803940 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:04:47.803940 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:04:47.803940 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:04:47.803940 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:04:47.803940 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:04:47.803940 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:04:47.803940 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 14 09:04:47.803940 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:04:47.803940 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:04:47.803940 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:04:47.803940 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:04:47.805335 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:04:47.805335 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:04:47.805335 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:04:47.805335 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:04:47.805335 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 09:04:47.805335 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 09:04:47.805335 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 09:04:47.805335 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 09:04:47.805335 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 09:04:47.805335 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 09:04:47.805335 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 09:04:47.805335 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 14 09:04:47.805335 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 247/484] 10.222.0.26 () {56 vars in 1102 bytes} [Thu Jul 14 09:04:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:04:47.810212 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406537d3-8b0d-4f13-ba7f-cefb926a4654 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:47.827284 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-c8606a6f-1cd6-43ea-af58-6c5549934cf4 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:04:47.830016 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-c8606a6f-1cd6-43ea-af58-6c5549934cf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:04:47.844767 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:47.964156 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 238/485] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:04:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 09:04:47.986968 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8606a6f-1cd6-43ea-af58-6c5549934cf4 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:48.005371 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406537d3-8b0d-4f13-ba7f-cefb926a4654 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:48.019404 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:48.039604 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccdc4b1-a98a-4c6f-8110-25a5b559f527 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:48.040277 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccdc4b1-a98a-4c6f-8110-25a5b559f527 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:48.040587 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccdc4b1-a98a-4c6f-8110-25a5b559f527 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:48.040811 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccdc4b1-a98a-4c6f-8110-25a5b559f527 None admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:48.041571 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccdc4b1-a98a-4c6f-8110-25a5b559f527 None admin] RBAC: Adding query filter params (name=service) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:04:48.041872 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccdc4b1-a98a-4c6f-8110-25a5b559f527 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:48.043280 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccdc4b1-a98a-4c6f-8110-25a5b559f527 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:48.052715 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 248/486] 10.222.0.26 () {56 vars in 1103 bytes} [Thu Jul 14 09:04:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:04:48.059846 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccdc4b1-a98a-4c6f-8110-25a5b559f527 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:48.166867 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8606a6f-1cd6-43ea-af58-6c5549934cf4 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:48.223871 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccdc4b1-a98a-4c6f-8110-25a5b559f527 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:48.232052 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:48.263703 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bdeca0-66a3-425c-858d-4671a19c5e1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:48.264491 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bdeca0-66a3-425c-858d-4671a19c5e1c None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:48.264771 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bdeca0-66a3-425c-858d-4671a19c5e1c None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:48.265041 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bdeca0-66a3-425c-858d-4671a19c5e1c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:48.285186 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be417b2a-8f16-4190-9254-a2d271d1429a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:48.286123 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bdeca0-66a3-425c-858d-4671a19c5e1c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 09:04:48.290410 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be417b2a-8f16-4190-9254-a2d271d1429a None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:48.290848 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be417b2a-8f16-4190-9254-a2d271d1429a None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:48.290940 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be417b2a-8f16-4190-9254-a2d271d1429a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:48.302939 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-31bdeca0-66a3-425c-858d-4671a19c5e1c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 09:04:48.302939 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:04:48.302939 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:04:48.302939 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:04:48.302939 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:04:48.302939 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:04:48.302939 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:04:48.302939 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:04:48.302939 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:04:48.302939 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:04:48.302939 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:04:48.302939 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:04:48.302939 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 09:04:48.302939 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 09:04:48.302939 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 09:04:48.302939 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 09:04:48.303898 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 09:04:48.303898 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 09:04:48.303898 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 09:04:48.303898 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 09:04:48.303898 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:04:48.303898 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:04:48.303898 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:04:48.303898 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:04:48.303898 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:04:48.303898 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:04:48.303898 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:04:48.303898 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 14 09:04:48.303898 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:04:48.303898 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:04:48.303898 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:04:48.303898 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:04:48.304711 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:04:48.304711 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:04:48.304711 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:04:48.304711 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:04:48.304711 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 09:04:48.304711 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 09:04:48.304711 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 09:04:48.304711 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 09:04:48.304711 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 09:04:48.304711 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 09:04:48.304711 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 09:04:48.304711 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 14 09:04:48.307162 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 239/487] 10.222.0.26 () {56 vars in 1102 bytes} [Thu Jul 14 09:04:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 321 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 09:04:48.308233 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be417b2a-8f16-4190-9254-a2d271d1429a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:48.313816 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be417b2a-8f16-4190-9254-a2d271d1429a None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:48.316320 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bdeca0-66a3-425c-858d-4671a19c5e1c None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:48.334323 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-be417b2a-8f16-4190-9254-a2d271d1429a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 09:04:48.334323 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:04:48.334323 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:04:48.334323 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:04:48.334323 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:04:48.334323 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:04:48.334323 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:04:48.334323 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:04:48.334323 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:04:48.334323 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:04:48.334323 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:04:48.334323 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:04:48.334323 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 09:04:48.334323 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 09:04:48.334323 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 09:04:48.335249 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 09:04:48.335249 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:04:48.335249 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:04:48.335249 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:04:48.335249 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:04:48.335249 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:04:48.335249 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:04:48.335249 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:04:48.335249 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 14 09:04:48.335249 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:04:48.335249 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:04:48.335249 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:04:48.335249 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:04:48.335249 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:04:48.335249 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:04:48.335249 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:04:48.335249 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:04:48.336125 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 09:04:48.336125 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 09:04:48.336125 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 09:04:48.336125 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 14 09:04:48.338085 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 249/488] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jul 14 09:04:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:04:48.348665 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be417b2a-8f16-4190-9254-a2d271d1429a None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:48.456546 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:48.474555 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bdeca0-66a3-425c-858d-4671a19c5e1c None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:48.505474 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be417b2a-8f16-4190-9254-a2d271d1429a None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:48.510436 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdb7d56-0347-41d8-b944-faef38f57807 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:48.511111 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdb7d56-0347-41d8-b944-faef38f57807 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:48.511427 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdb7d56-0347-41d8-b944-faef38f57807 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:48.511704 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdb7d56-0347-41d8-b944-faef38f57807 None admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:48.512461 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdb7d56-0347-41d8-b944-faef38f57807 None admin] RBAC: Adding query filter params (name=service) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:04:48.512770 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdb7d56-0347-41d8-b944-faef38f57807 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:48.514399 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdb7d56-0347-41d8-b944-faef38f57807 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:48.522954 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 240/489] 10.222.0.26 () {56 vars in 1103 bytes} [Thu Jul 14 09:04:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:04:48.524216 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7bdd48-de55-48a8-b296-1fc974cdf93a None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:48.524511 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7bdd48-de55-48a8-b296-1fc974cdf93a None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:48.524790 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7bdd48-de55-48a8-b296-1fc974cdf93a None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:48.526148 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 241/490] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:04:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:04:48.527352 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c7eee7-e936-4458-ab91-509917839d11 None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:48.527641 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c7eee7-e936-4458-ab91-509917839d11 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:48.527919 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c7eee7-e936-4458-ab91-509917839d11 None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:48.533155 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 242/491] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:04:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:04:48.533155 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c7eee7-e936-4458-ab91-509917839d11 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:48.542424 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:48.555221 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9896bb46-0628-422a-8e3a-7a70cbee8e6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:48.555897 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9896bb46-0628-422a-8e3a-7a70cbee8e6f None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:48.556180 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9896bb46-0628-422a-8e3a-7a70cbee8e6f None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:48.556451 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9896bb46-0628-422a-8e3a-7a70cbee8e6f None admin] PATH_INFO: `/v3/domains` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:48.557302 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9896bb46-0628-422a-8e3a-7a70cbee8e6f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:04:48.557718 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9896bb46-0628-422a-8e3a-7a70cbee8e6f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:48.559095 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9896bb46-0628-422a-8e3a-7a70cbee8e6f None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:48.573384 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 250/492] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Jul 14 09:04:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:04:48.578971 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf7eead-f300-4d89-82fe-4cfe174b0154 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:48.579280 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf7eead-f300-4d89-82fe-4cfe174b0154 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:48.579556 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf7eead-f300-4d89-82fe-4cfe174b0154 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:48.596477 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-4cf7eead-f300-4d89-82fe-4cfe174b0154 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:04:48.597776 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-4cf7eead-f300-4d89-82fe-4cfe174b0154 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:04:48.641025 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c7eee7-e936-4458-ab91-509917839d11 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:48.668093 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 251/493] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:04:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 09:04:48.669978 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d41041-0a6d-4cd4-8c5b-777208e712c8 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:48.670245 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d41041-0a6d-4cd4-8c5b-777208e712c8 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:48.670520 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d41041-0a6d-4cd4-8c5b-777208e712c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:48.686577 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0591ca4-423c-49ea-9277-e9e7c4c97728 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:48.687681 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0591ca4-423c-49ea-9277-e9e7c4c97728 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:48.688168 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0591ca4-423c-49ea-9277-e9e7c4c97728 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:48.688726 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0591ca4-423c-49ea-9277-e9e7c4c97728 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:48.689369 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-44d41041-0a6d-4cd4-8c5b-777208e712c8 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:04:48.690607 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-44d41041-0a6d-4cd4-8c5b-777208e712c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:04:48.702868 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0591ca4-423c-49ea-9277-e9e7c4c97728 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:48.704641 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0591ca4-423c-49ea-9277-e9e7c4c97728 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:48.713211 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:48.725595 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-d0591ca4-423c-49ea-9277-e9e7c4c97728 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 09:04:48.725595 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:04:48.725595 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:04:48.725595 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:04:48.725595 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:04:48.725595 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:04:48.725595 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:04:48.725595 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:04:48.725595 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:04:48.725595 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:04:48.725595 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:04:48.725595 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:04:48.725595 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 09:04:48.725595 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 09:04:48.725595 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 09:04:48.726464 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 09:04:48.726464 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:04:48.726464 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:04:48.726464 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:04:48.726464 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:04:48.726464 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:04:48.726464 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:04:48.726464 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:04:48.726464 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 14 09:04:48.726464 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:04:48.726464 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:04:48.726464 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:04:48.726464 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:04:48.726464 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:04:48.726464 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:04:48.726464 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:04:48.726464 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:04:48.727749 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 09:04:48.727749 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 09:04:48.727749 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 09:04:48.727749 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 14 09:04:48.727749 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 243/494] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jul 14 09:04:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:04:48.727749 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0591ca4-423c-49ea-9277-e9e7c4c97728 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:48.771937 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 252/495] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:04:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 09:04:48.773664 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d41041-0a6d-4cd4-8c5b-777208e712c8 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:48.829396 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:48.839310 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0591ca4-423c-49ea-9277-e9e7c4c97728 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:48.875132 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6e90f7-a604-4910-9d52-015f5e1f811f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:48.875966 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6e90f7-a604-4910-9d52-015f5e1f811f None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:48.876270 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6e90f7-a604-4910-9d52-015f5e1f811f None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:48.876549 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6e90f7-a604-4910-9d52-015f5e1f811f None admin] PATH_INFO: `/v3/users/nova` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:48.883152 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d41041-0a6d-4cd4-8c5b-777208e712c8 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:48.884035 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6e90f7-a604-4910-9d52-015f5e1f811f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:48.885492 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6e90f7-a604-4910-9d52-015f5e1f811f None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:48.901315 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-8c6e90f7-a604-4910-9d52-015f5e1f811f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 14 09:04:48.901315 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:04:48.901315 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:04:48.901315 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:04:48.901315 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:04:48.901315 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:04:48.901315 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:04:48.901315 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:04:48.901315 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:04:48.901315 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:04:48.901315 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:04:48.901315 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:04:48.901315 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 09:04:48.901315 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 09:04:48.901315 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 09:04:48.901315 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 09:04:48.902432 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:04:48.902432 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:04:48.902432 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 09:04:48.902432 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 09:04:48.902432 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 09:04:48.902432 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 09:04:48.902432 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:04:48.902432 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:04:48.902432 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:04:48.902432 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:04:48.902432 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:04:48.902432 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 14 09:04:48.902432 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:04:48.902432 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:04:48.902432 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:04:48.902432 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:04:48.902432 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:04:48.902432 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:04:48.903400 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:04:48.903400 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:04:48.903400 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 09:04:48.903400 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 09:04:48.903400 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 09:04:48.903400 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 09:04:48.903400 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 09:04:48.903400 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 09:04:48.903400 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 14 09:04:48.903400 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 14 09:04:48.903400 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 244/496] 10.222.0.26 () {56 vars in 1093 bytes} [Thu Jul 14 09:04:48 2022] GET /identity/v3/users/nova => generated 82 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:04:48.903400 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6e90f7-a604-4910-9d52-015f5e1f811f None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:48.922293 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6df0a7-2993-4566-8bb8-5a2cace98aed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:48.923363 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6df0a7-2993-4566-8bb8-5a2cace98aed None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:48.923889 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6df0a7-2993-4566-8bb8-5a2cace98aed None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:48.924324 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6df0a7-2993-4566-8bb8-5a2cace98aed None admin] PATH_INFO: `/v3/roles/service` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:48.935841 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6df0a7-2993-4566-8bb8-5a2cace98aed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 09:04:48.944023 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-bb6df0a7-2993-4566-8bb8-5a2cace98aed None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 09:04:48.944023 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:04:48.944023 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:04:48.944023 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:04:48.944023 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:04:48.944023 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:04:48.944023 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:04:48.944023 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:04:48.944023 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:04:48.944023 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:04:48.944023 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:04:48.944023 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:04:48.944023 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 09:04:48.944023 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 09:04:48.944023 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 09:04:48.944023 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 09:04:48.945094 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 09:04:48.945094 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 09:04:48.945094 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 09:04:48.945094 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 09:04:48.945094 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:04:48.945094 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:04:48.945094 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:04:48.945094 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:04:48.945094 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:04:48.945094 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:04:48.945094 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:04:48.945094 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 14 09:04:48.945094 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:04:48.945094 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:04:48.945094 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:04:48.945094 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:04:48.946049 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:04:48.946049 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:04:48.946049 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:04:48.946049 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:04:48.946049 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 09:04:48.946049 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 09:04:48.946049 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 09:04:48.946049 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 09:04:48.946049 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 09:04:48.946049 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 09:04:48.946049 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 09:04:48.946049 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 14 09:04:48.948173 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 253/497] 10.222.0.26 () {56 vars in 1102 bytes} [Thu Jul 14 09:04:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:04:48.953734 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6df0a7-2993-4566-8bb8-5a2cace98aed None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:48.964346 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:49.006920 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6e90f7-a604-4910-9d52-015f5e1f811f None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:49.039174 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb97bb2-fdbb-41e5-96f0-ed97c8bc04b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:49.039792 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb97bb2-fdbb-41e5-96f0-ed97c8bc04b5 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:49.040075 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb97bb2-fdbb-41e5-96f0-ed97c8bc04b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:49.040351 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb97bb2-fdbb-41e5-96f0-ed97c8bc04b5 None admin] PATH_INFO: `/v3/domains` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:49.041093 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb97bb2-fdbb-41e5-96f0-ed97c8bc04b5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:04:49.041407 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb97bb2-fdbb-41e5-96f0-ed97c8bc04b5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:49.042800 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb97bb2-fdbb-41e5-96f0-ed97c8bc04b5 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:49.056106 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6df0a7-2993-4566-8bb8-5a2cace98aed None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:49.056714 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 245/498] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Jul 14 09:04:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:04:49.057592 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb97bb2-fdbb-41e5-96f0-ed97c8bc04b5 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:49.066606 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:49.093837 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1eb254-b2b1-41fe-8f6f-7b718766f121 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:49.094952 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1eb254-b2b1-41fe-8f6f-7b718766f121 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:49.095412 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1eb254-b2b1-41fe-8f6f-7b718766f121 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:49.095816 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1eb254-b2b1-41fe-8f6f-7b718766f121 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:49.107443 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1eb254-b2b1-41fe-8f6f-7b718766f121 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 09:04:49.116866 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-db1eb254-b2b1-41fe-8f6f-7b718766f121 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 09:04:49.116866 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:04:49.116866 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:04:49.116866 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:04:49.116866 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:04:49.116866 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:04:49.116866 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:04:49.116866 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:04:49.116866 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:04:49.116866 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:04:49.116866 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:04:49.116866 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:04:49.116866 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 09:04:49.116866 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 09:04:49.116866 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 09:04:49.116866 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 09:04:49.117767 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 09:04:49.117767 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 09:04:49.117767 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 09:04:49.117767 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 09:04:49.117767 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:04:49.117767 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:04:49.117767 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:04:49.117767 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:04:49.117767 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:04:49.117767 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:04:49.117767 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:04:49.117767 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 14 09:04:49.117767 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:04:49.117767 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:04:49.117767 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:04:49.117767 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:04:49.118573 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:04:49.118573 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:04:49.118573 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:04:49.118573 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:04:49.118573 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 09:04:49.118573 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 09:04:49.118573 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 09:04:49.118573 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 09:04:49.118573 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 09:04:49.118573 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 09:04:49.118573 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 09:04:49.118573 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 14 09:04:49.118573 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 254/499] 10.222.0.26 () {56 vars in 1102 bytes} [Thu Jul 14 09:04:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:04:49.119260 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1eb254-b2b1-41fe-8f6f-7b718766f121 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:49.164143 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb97bb2-fdbb-41e5-96f0-ed97c8bc04b5 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:49.181910 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:49.199580 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fd5cc3-66fe-4e9f-a2f3-092699a620ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:49.200247 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fd5cc3-66fe-4e9f-a2f3-092699a620ed None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:49.200532 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fd5cc3-66fe-4e9f-a2f3-092699a620ed None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:49.200808 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fd5cc3-66fe-4e9f-a2f3-092699a620ed None admin] PATH_INFO: `/v3/users` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:49.201725 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fd5cc3-66fe-4e9f-a2f3-092699a620ed None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:04:49.202034 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fd5cc3-66fe-4e9f-a2f3-092699a620ed None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:49.203495 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fd5cc3-66fe-4e9f-a2f3-092699a620ed None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:49.220792 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1eb254-b2b1-41fe-8f6f-7b718766f121 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:49.238985 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 246/500] 10.222.0.26 () {56 vars in 1133 bytes} [Thu Jul 14 09:04:49 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:04:49.240318 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c003ab1-74d6-4cb1-a6ac-8ebe034a77d2 None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:49.240602 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c003ab1-74d6-4cb1-a6ac-8ebe034a77d2 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:49.240877 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c003ab1-74d6-4cb1-a6ac-8ebe034a77d2 None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:49.242285 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 247/501] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:04:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:04:49.242773 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c003ab1-74d6-4cb1-a6ac-8ebe034a77d2 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:49.257683 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f949c6-c484-4bf0-9bc9-5ac3f4b63c6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:49.258356 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f949c6-c484-4bf0-9bc9-5ac3f4b63c6f None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:49.258639 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f949c6-c484-4bf0-9bc9-5ac3f4b63c6f None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:49.258920 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f949c6-c484-4bf0-9bc9-5ac3f4b63c6f None admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:49.259722 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f949c6-c484-4bf0-9bc9-5ac3f4b63c6f None admin] RBAC: Adding query filter params (name=service) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:04:49.260032 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f949c6-c484-4bf0-9bc9-5ac3f4b63c6f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:49.261429 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f949c6-c484-4bf0-9bc9-5ac3f4b63c6f None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:49.269428 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 255/502] 10.222.0.26 () {56 vars in 1103 bytes} [Thu Jul 14 09:04:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:04:49.270197 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f949c6-c484-4bf0-9bc9-5ac3f4b63c6f None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:49.303925 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:49.345359 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c003ab1-74d6-4cb1-a6ac-8ebe034a77d2 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:49.371677 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f949c6-c484-4bf0-9bc9-5ac3f4b63c6f None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:49.385116 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7281801-0e55-4c6d-aecd-a6a9bf988f94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:49.385911 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7281801-0e55-4c6d-aecd-a6a9bf988f94 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:49.386194 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7281801-0e55-4c6d-aecd-a6a9bf988f94 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:49.386468 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7281801-0e55-4c6d-aecd-a6a9bf988f94 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:49.393756 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7281801-0e55-4c6d-aecd-a6a9bf988f94 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:49.395266 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7281801-0e55-4c6d-aecd-a6a9bf988f94 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:49.403569 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00720049-f620-4eda-95fb-d354e6175a3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:49.404303 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00720049-f620-4eda-95fb-d354e6175a3f None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:49.404591 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00720049-f620-4eda-95fb-d354e6175a3f None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:49.404864 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00720049-f620-4eda-95fb-d354e6175a3f None admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:49.405608 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00720049-f620-4eda-95fb-d354e6175a3f None admin] RBAC: Adding query filter params (name=service) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:04:49.405909 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00720049-f620-4eda-95fb-d354e6175a3f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:49.407310 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00720049-f620-4eda-95fb-d354e6175a3f None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:49.411970 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-a7281801-0e55-4c6d-aecd-a6a9bf988f94 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 14 09:04:49.411970 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:04:49.411970 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:04:49.411970 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:04:49.411970 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:04:49.411970 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:04:49.411970 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:04:49.411970 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:04:49.411970 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:04:49.411970 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:04:49.411970 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:04:49.411970 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:04:49.411970 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 09:04:49.411970 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 09:04:49.411970 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 09:04:49.411970 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 09:04:49.412985 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:04:49.412985 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:04:49.412985 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 09:04:49.412985 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 09:04:49.412985 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 09:04:49.412985 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 09:04:49.412985 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:04:49.412985 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:04:49.412985 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:04:49.412985 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:04:49.412985 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:04:49.412985 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 14 09:04:49.412985 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:04:49.412985 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:04:49.412985 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:04:49.412985 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:04:49.412985 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:04:49.412985 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:04:49.414047 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:04:49.414047 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:04:49.414047 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 09:04:49.414047 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 09:04:49.414047 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 09:04:49.414047 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 09:04:49.414047 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 09:04:49.414047 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 09:04:49.414047 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 14 09:04:49.414047 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 14 09:04:49.414047 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 248/503] 10.222.0.26 () {56 vars in 1099 bytes} [Thu Jul 14 09:04:49 2022] GET /identity/v3/users/glance => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:04:49.414047 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7281801-0e55-4c6d-aecd-a6a9bf988f94 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:49.418417 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 256/504] 10.222.0.26 () {56 vars in 1103 bytes} [Thu Jul 14 09:04:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:04:49.419689 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fd00e3-033b-4f68-bda8-37a7aaf49923 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:49.420080 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fd00e3-033b-4f68-bda8-37a7aaf49923 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:49.420339 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fd00e3-033b-4f68-bda8-37a7aaf49923 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:49.436931 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-c9fd00e3-033b-4f68-bda8-37a7aaf49923 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:04:49.438067 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-c9fd00e3-033b-4f68-bda8-37a7aaf49923 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:04:49.452778 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:49.506396 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 257/505] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:04:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 09:04:49.507258 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fd00e3-033b-4f68-bda8-37a7aaf49923 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:49.527785 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7281801-0e55-4c6d-aecd-a6a9bf988f94 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:49.533756 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:49.568711 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff03c9a-6d8c-47df-a5f4-eb52f60e1af8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:49.569291 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff03c9a-6d8c-47df-a5f4-eb52f60e1af8 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:49.569576 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff03c9a-6d8c-47df-a5f4-eb52f60e1af8 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:49.569851 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff03c9a-6d8c-47df-a5f4-eb52f60e1af8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:49.613350 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fd00e3-033b-4f68-bda8-37a7aaf49923 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:49.638133 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff03c9a-6d8c-47df-a5f4-eb52f60e1af8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:49.639767 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff03c9a-6d8c-47df-a5f4-eb52f60e1af8 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:49.645966 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4cd11e-b794-410c-bafa-ba6177a66fe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:49.646787 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4cd11e-b794-410c-bafa-ba6177a66fe2 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:49.647072 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4cd11e-b794-410c-bafa-ba6177a66fe2 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:49.647381 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4cd11e-b794-410c-bafa-ba6177a66fe2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:49.657968 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4cd11e-b794-410c-bafa-ba6177a66fe2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:49.659478 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4cd11e-b794-410c-bafa-ba6177a66fe2 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:49.674352 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:49.677917 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-5b4cd11e-b794-410c-bafa-ba6177a66fe2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 09:04:49.677917 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:04:49.677917 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:04:49.677917 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:04:49.677917 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:04:49.677917 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:04:49.677917 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:04:49.677917 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:04:49.677917 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:04:49.677917 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:04:49.677917 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:04:49.677917 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:04:49.677917 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 09:04:49.677917 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 09:04:49.677917 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 09:04:49.678918 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 09:04:49.678918 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:04:49.678918 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:04:49.678918 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:04:49.678918 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:04:49.678918 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:04:49.678918 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:04:49.678918 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:04:49.678918 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 14 09:04:49.678918 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:04:49.678918 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:04:49.678918 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:04:49.678918 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:04:49.678918 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:04:49.678918 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:04:49.678918 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:04:49.678918 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:04:49.680002 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 09:04:49.680002 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 09:04:49.680002 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 09:04:49.680002 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 14 09:04:49.680002 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 258/506] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jul 14 09:04:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:04:49.681843 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4cd11e-b794-410c-bafa-ba6177a66fe2 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:49.740657 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 249/507] 10.222.0.26 () {58 vars in 1282 bytes} [Thu Jul 14 09:04:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 09:04:49.741578 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff03c9a-6d8c-47df-a5f4-eb52f60e1af8 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:49.756174 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:49.785186 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4cd11e-b794-410c-bafa-ba6177a66fe2 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:49.819322 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf87d12a-89cc-480e-b11a-ad4745c5aa62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:49.819926 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf87d12a-89cc-480e-b11a-ad4745c5aa62 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:49.820211 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf87d12a-89cc-480e-b11a-ad4745c5aa62 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:49.820483 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf87d12a-89cc-480e-b11a-ad4745c5aa62 None admin] PATH_INFO: `/v3/users` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:49.821327 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf87d12a-89cc-480e-b11a-ad4745c5aa62 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:04:49.821633 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf87d12a-89cc-480e-b11a-ad4745c5aa62 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:49.823284 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf87d12a-89cc-480e-b11a-ad4745c5aa62 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:49.846414 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff03c9a-6d8c-47df-a5f4-eb52f60e1af8 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:49.858894 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 259/508] 10.222.0.26 () {56 vars in 1137 bytes} [Thu Jul 14 09:04:49 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:04:49.859970 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf87d12a-89cc-480e-b11a-ad4745c5aa62 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:49.882771 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e592f76-60ac-44ac-8eb1-89951622e207 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:49.883603 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e592f76-60ac-44ac-8eb1-89951622e207 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:49.883883 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e592f76-60ac-44ac-8eb1-89951622e207 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:49.884153 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e592f76-60ac-44ac-8eb1-89951622e207 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:49.886899 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:49.895306 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e592f76-60ac-44ac-8eb1-89951622e207 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:49.896765 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e592f76-60ac-44ac-8eb1-89951622e207 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:49.914248 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-0e592f76-60ac-44ac-8eb1-89951622e207 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 09:04:49.914248 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:04:49.914248 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:04:49.914248 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:04:49.914248 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:04:49.914248 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:04:49.914248 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:04:49.914248 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:04:49.914248 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:04:49.914248 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:04:49.914248 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:04:49.914248 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:04:49.914248 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 09:04:49.914248 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 09:04:49.914248 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 09:04:49.915106 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 09:04:49.915106 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:04:49.915106 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:04:49.915106 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:04:49.915106 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:04:49.915106 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:04:49.915106 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:04:49.915106 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:04:49.915106 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 14 09:04:49.915106 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:04:49.915106 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:04:49.915106 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:04:49.915106 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:04:49.915106 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:04:49.915106 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:04:49.915106 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:04:49.915106 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:04:49.916206 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 09:04:49.916206 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 09:04:49.916206 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 09:04:49.916206 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 14 09:04:49.916206 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 250/509] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jul 14 09:04:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:04:49.916602 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e592f76-60ac-44ac-8eb1-89951622e207 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:49.967457 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf87d12a-89cc-480e-b11a-ad4745c5aa62 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:49.995034 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:50.001200 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31d7214-d167-4076-957f-2521c83268df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:50.002007 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31d7214-d167-4076-957f-2521c83268df None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:50.002292 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31d7214-d167-4076-957f-2521c83268df None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:50.002572 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31d7214-d167-4076-957f-2521c83268df None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:50.012955 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31d7214-d167-4076-957f-2521c83268df None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:50.014461 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31d7214-d167-4076-957f-2521c83268df None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:50.017743 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e592f76-60ac-44ac-8eb1-89951622e207 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:50.032076 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-d31d7214-d167-4076-957f-2521c83268df None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 09:04:50.032076 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:04:50.032076 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:04:50.032076 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:04:50.032076 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:04:50.032076 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:04:50.032076 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:04:50.032076 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:04:50.032076 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:04:50.032076 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:04:50.032076 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:04:50.032076 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:04:50.032076 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 09:04:50.032076 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 09:04:50.032076 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 09:04:50.032955 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 09:04:50.032955 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:04:50.032955 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:04:50.032955 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:04:50.032955 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:04:50.032955 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:04:50.032955 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:04:50.032955 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:04:50.032955 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 14 09:04:50.032955 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:04:50.032955 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:04:50.032955 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:04:50.032955 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:04:50.032955 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:04:50.032955 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:04:50.032955 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:04:50.032955 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:04:50.033799 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 09:04:50.033799 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 09:04:50.033799 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 09:04:50.033799 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 14 09:04:50.035357 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 260/510] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jul 14 09:04:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:04:50.036453 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31d7214-d167-4076-957f-2521c83268df None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:50.050638 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe00198d-63fd-48c1-b355-ca1bc488fbd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:50.051327 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe00198d-63fd-48c1-b355-ca1bc488fbd6 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:50.051609 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe00198d-63fd-48c1-b355-ca1bc488fbd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:50.051884 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe00198d-63fd-48c1-b355-ca1bc488fbd6 None admin] PATH_INFO: `/v3/domains` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:50.052609 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe00198d-63fd-48c1-b355-ca1bc488fbd6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:04:50.053034 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe00198d-63fd-48c1-b355-ca1bc488fbd6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:50.054385 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe00198d-63fd-48c1-b355-ca1bc488fbd6 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:50.072859 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 251/511] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Jul 14 09:04:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:04:50.072859 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe00198d-63fd-48c1-b355-ca1bc488fbd6 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:50.120572 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:50.145694 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31d7214-d167-4076-957f-2521c83268df None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:50.171980 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe00198d-63fd-48c1-b355-ca1bc488fbd6 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:50.178967 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8332fabc-94d9-4537-b258-fba320df610a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:50.179792 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8332fabc-94d9-4537-b258-fba320df610a None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:50.180076 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8332fabc-94d9-4537-b258-fba320df610a None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:50.180342 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8332fabc-94d9-4537-b258-fba320df610a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:50.192188 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8332fabc-94d9-4537-b258-fba320df610a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:50.193665 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8332fabc-94d9-4537-b258-fba320df610a None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:50.205482 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3952d9ee-d1df-4f55-b134-a9e179d0ae11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:50.206120 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3952d9ee-d1df-4f55-b134-a9e179d0ae11 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:50.206405 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3952d9ee-d1df-4f55-b134-a9e179d0ae11 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:50.206727 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3952d9ee-d1df-4f55-b134-a9e179d0ae11 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:50.210244 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-8332fabc-94d9-4537-b258-fba320df610a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 09:04:50.210244 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:04:50.210244 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:04:50.210244 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:04:50.210244 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:04:50.210244 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:04:50.210244 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:04:50.210244 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:04:50.210244 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:04:50.210244 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:04:50.210244 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:04:50.210244 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:04:50.210244 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 09:04:50.210244 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 09:04:50.210244 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 09:04:50.211093 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 09:04:50.211093 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:04:50.211093 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:04:50.211093 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:04:50.211093 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:04:50.211093 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:04:50.211093 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:04:50.211093 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:04:50.211093 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 14 09:04:50.211093 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:04:50.211093 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:04:50.211093 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:04:50.211093 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:04:50.211093 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:04:50.211093 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:04:50.211093 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:04:50.211093 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:04:50.211967 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 09:04:50.211967 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 09:04:50.211967 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 09:04:50.211967 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 14 09:04:50.211967 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 261/512] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jul 14 09:04:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:04:50.212241 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8332fabc-94d9-4537-b258-fba320df610a None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:50.248033 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:50.275802 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3952d9ee-d1df-4f55-b134-a9e179d0ae11 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:50.277273 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3952d9ee-d1df-4f55-b134-a9e179d0ae11 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:50.312900 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8332fabc-94d9-4537-b258-fba320df610a None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:50.323747 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:50.345622 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39146507-351c-4628-a3c5-b5843450686a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:50.346280 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39146507-351c-4628-a3c5-b5843450686a None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:50.346564 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39146507-351c-4628-a3c5-b5843450686a None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:50.346839 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39146507-351c-4628-a3c5-b5843450686a None admin] PATH_INFO: `/v3/domains` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:50.347598 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39146507-351c-4628-a3c5-b5843450686a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:04:50.347905 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39146507-351c-4628-a3c5-b5843450686a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:50.349277 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39146507-351c-4628-a3c5-b5843450686a None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:50.361058 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 262/513] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Jul 14 09:04:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:04:50.362244 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39146507-351c-4628-a3c5-b5843450686a None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:50.374128 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 252/514] 10.222.0.26 () {58 vars in 1282 bytes} [Thu Jul 14 09:04:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 09:04:50.375024 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3952d9ee-d1df-4f55-b134-a9e179d0ae11 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:50.461097 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:50.467739 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39146507-351c-4628-a3c5-b5843450686a None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:50.476393 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3952d9ee-d1df-4f55-b134-a9e179d0ae11 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:50.501642 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2d2b17-7863-476d-bfac-3a0965a9bf5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:50.502309 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2d2b17-7863-476d-bfac-3a0965a9bf5f None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:50.502588 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2d2b17-7863-476d-bfac-3a0965a9bf5f None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:50.502860 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2d2b17-7863-476d-bfac-3a0965a9bf5f None admin] PATH_INFO: `/v3/domains` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:50.503620 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2d2b17-7863-476d-bfac-3a0965a9bf5f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:04:50.503924 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2d2b17-7863-476d-bfac-3a0965a9bf5f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:50.505338 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2d2b17-7863-476d-bfac-3a0965a9bf5f None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:50.509139 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc047f1f-32b9-4198-8adb-35e1722b8e2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:50.509931 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc047f1f-32b9-4198-8adb-35e1722b8e2d None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:50.510208 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc047f1f-32b9-4198-8adb-35e1722b8e2d None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:50.510477 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc047f1f-32b9-4198-8adb-35e1722b8e2d None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:50.517036 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 263/515] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Jul 14 09:04:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:04:50.518218 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc047f1f-32b9-4198-8adb-35e1722b8e2d None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:50.518391 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2d2b17-7863-476d-bfac-3a0965a9bf5f None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:50.519683 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc047f1f-32b9-4198-8adb-35e1722b8e2d None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:50.534561 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-cc047f1f-32b9-4198-8adb-35e1722b8e2d None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 14 09:04:50.534561 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:04:50.534561 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:04:50.534561 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:04:50.534561 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:04:50.534561 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:04:50.534561 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:04:50.534561 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:04:50.534561 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:04:50.534561 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:04:50.534561 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:04:50.534561 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:04:50.534561 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 09:04:50.534561 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 09:04:50.534561 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 09:04:50.534561 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 09:04:50.535959 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:04:50.535959 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:04:50.535959 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 09:04:50.535959 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 09:04:50.535959 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 09:04:50.535959 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 09:04:50.535959 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:04:50.535959 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:04:50.535959 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:04:50.535959 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:04:50.535959 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:04:50.535959 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 14 09:04:50.535959 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:04:50.535959 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:04:50.535959 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:04:50.535959 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:04:50.535959 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:04:50.535959 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:04:50.536855 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:04:50.536855 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:04:50.536855 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 09:04:50.536855 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 09:04:50.536855 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 09:04:50.536855 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 09:04:50.536855 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 09:04:50.536855 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 09:04:50.536855 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 14 09:04:50.536855 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 14 09:04:50.536855 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 253/516] 10.222.0.26 () {56 vars in 1099 bytes} [Thu Jul 14 09:04:50 2022] GET /identity/v3/users/cinder => generated 84 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:04:50.536855 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc047f1f-32b9-4198-8adb-35e1722b8e2d None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:50.548496 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:50.624153 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2d2b17-7863-476d-bfac-3a0965a9bf5f None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:50.635684 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc047f1f-32b9-4198-8adb-35e1722b8e2d None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:50.656029 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79185ad2-6e07-4d8d-a0d0-c0049700c7c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:50.656687 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79185ad2-6e07-4d8d-a0d0-c0049700c7c5 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:50.656966 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79185ad2-6e07-4d8d-a0d0-c0049700c7c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:50.657237 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79185ad2-6e07-4d8d-a0d0-c0049700c7c5 None admin] PATH_INFO: `/v3/domains` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:50.657957 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79185ad2-6e07-4d8d-a0d0-c0049700c7c5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:04:50.658258 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79185ad2-6e07-4d8d-a0d0-c0049700c7c5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:50.659656 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79185ad2-6e07-4d8d-a0d0-c0049700c7c5 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:50.670615 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da80046c-0048-49ad-9fa6-992c36e6066e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:50.671440 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da80046c-0048-49ad-9fa6-992c36e6066e None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:50.671730 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da80046c-0048-49ad-9fa6-992c36e6066e None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:50.671946 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 264/517] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Jul 14 09:04:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:04:50.672050 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da80046c-0048-49ad-9fa6-992c36e6066e None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:50.672770 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79185ad2-6e07-4d8d-a0d0-c0049700c7c5 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:50.680099 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da80046c-0048-49ad-9fa6-992c36e6066e None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:50.681548 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da80046c-0048-49ad-9fa6-992c36e6066e None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:50.691404 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:50.696039 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-da80046c-0048-49ad-9fa6-992c36e6066e None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 14 09:04:50.696039 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:04:50.696039 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:04:50.696039 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:04:50.696039 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:04:50.696039 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:04:50.696039 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:04:50.696039 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:04:50.696039 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:04:50.696039 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:04:50.696039 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:04:50.696039 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:04:50.696039 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 09:04:50.696039 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 09:04:50.696039 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 09:04:50.696039 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 09:04:50.696947 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:04:50.696947 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:04:50.696947 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 09:04:50.696947 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 09:04:50.696947 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 09:04:50.696947 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 09:04:50.696947 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:04:50.696947 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:04:50.696947 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:04:50.696947 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:04:50.696947 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:04:50.696947 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 14 09:04:50.696947 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:04:50.696947 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:04:50.696947 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:04:50.696947 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:04:50.696947 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:04:50.696947 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:04:50.697843 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:04:50.697843 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:04:50.697843 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 09:04:50.697843 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 09:04:50.697843 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 09:04:50.697843 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 09:04:50.697843 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 09:04:50.697843 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 09:04:50.697843 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 14 09:04:50.697843 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 14 09:04:50.697843 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 254/518] 10.222.0.26 () {56 vars in 1102 bytes} [Thu Jul 14 09:04:50 2022] GET /identity/v3/users/neutron => generated 85 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:04:50.698392 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da80046c-0048-49ad-9fa6-992c36e6066e None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:50.765652 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:50.775921 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79185ad2-6e07-4d8d-a0d0-c0049700c7c5 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:50.801215 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da80046c-0048-49ad-9fa6-992c36e6066e None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:50.813521 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d69410-a8f4-470c-9a2a-cf81aa8edf3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:50.814510 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d69410-a8f4-470c-9a2a-cf81aa8edf3d None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:50.814941 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d69410-a8f4-470c-9a2a-cf81aa8edf3d None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:50.815238 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d69410-a8f4-470c-9a2a-cf81aa8edf3d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:50.825497 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d69410-a8f4-470c-9a2a-cf81aa8edf3d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:50.826990 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d69410-a8f4-470c-9a2a-cf81aa8edf3d None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:50.833846 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d24a2b-e1a1-4e31-86e0-1fc58d6cc8ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:50.834513 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d24a2b-e1a1-4e31-86e0-1fc58d6cc8ec None admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:50.834800 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d24a2b-e1a1-4e31-86e0-1fc58d6cc8ec None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:50.835076 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d24a2b-e1a1-4e31-86e0-1fc58d6cc8ec None admin] PATH_INFO: `/v3/users` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:50.835814 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d24a2b-e1a1-4e31-86e0-1fc58d6cc8ec None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:50.837246 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d24a2b-e1a1-4e31-86e0-1fc58d6cc8ec None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:50.844669 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-30d69410-a8f4-470c-9a2a-cf81aa8edf3d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 09:04:50.844669 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:04:50.844669 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:04:50.844669 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:04:50.844669 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:04:50.844669 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:04:50.844669 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:04:50.844669 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:04:50.844669 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:04:50.844669 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:04:50.844669 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:04:50.844669 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:04:50.844669 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 09:04:50.844669 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 09:04:50.844669 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 09:04:50.845541 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 09:04:50.845541 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:04:50.845541 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:04:50.845541 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:04:50.845541 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:04:50.845541 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:04:50.845541 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:04:50.845541 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:04:50.845541 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 14 09:04:50.845541 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:04:50.845541 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:04:50.845541 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:04:50.845541 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:04:50.845541 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:04:50.845541 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:04:50.845541 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:04:50.845541 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:04:50.846395 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 09:04:50.846395 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 09:04:50.846395 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 09:04:50.846395 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 14 09:04:50.846395 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 265/519] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jul 14 09:04:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:04:50.846995 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d69410-a8f4-470c-9a2a-cf81aa8edf3d None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:50.861466 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 255/520] 10.222.0.26 () {60 vars in 1132 bytes} [Thu Jul 14 09:04:50 2022] POST /identity/v3/users => generated 247 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 09:04:50.862709 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d24a2b-e1a1-4e31-86e0-1fc58d6cc8ec None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:50.932472 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:50.969728 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d69410-a8f4-470c-9a2a-cf81aa8edf3d None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:50.977401 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d24a2b-e1a1-4e31-86e0-1fc58d6cc8ec None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:51.003411 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:51.009003 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8f93d7-39ee-4765-a35d-184c95c5cb0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:51.009697 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8f93d7-39ee-4765-a35d-184c95c5cb0d None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:51.009972 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8f93d7-39ee-4765-a35d-184c95c5cb0d None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:51.010249 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8f93d7-39ee-4765-a35d-184c95c5cb0d None admin] PATH_INFO: `/v3/users` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:51.011110 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8f93d7-39ee-4765-a35d-184c95c5cb0d None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:04:51.012169 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8f93d7-39ee-4765-a35d-184c95c5cb0d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:51.013224 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8f93d7-39ee-4765-a35d-184c95c5cb0d None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:51.018402 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d7e934-286a-412f-9619-b75d259fec38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:51.019393 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d7e934-286a-412f-9619-b75d259fec38 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:51.020011 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d7e934-286a-412f-9619-b75d259fec38 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:51.020011 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d7e934-286a-412f-9619-b75d259fec38 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:51.028511 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d7e934-286a-412f-9619-b75d259fec38 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:51.030045 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d7e934-286a-412f-9619-b75d259fec38 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:51.032104 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 266/521] 10.222.0.26 () {56 vars in 1137 bytes} [Thu Jul 14 09:04:50 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:04:51.032758 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8f93d7-39ee-4765-a35d-184c95c5cb0d None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:51.048363 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-38d7e934-286a-412f-9619-b75d259fec38 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 09:04:51.048363 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:04:51.048363 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:04:51.048363 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:04:51.048363 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:04:51.048363 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:04:51.048363 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:04:51.048363 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:04:51.048363 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:04:51.048363 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:04:51.048363 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:04:51.048363 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:04:51.048363 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 09:04:51.048363 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 09:04:51.048363 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 09:04:51.049257 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 09:04:51.049257 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:04:51.049257 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:04:51.049257 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:04:51.049257 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:04:51.049257 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:04:51.049257 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:04:51.049257 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:04:51.049257 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 14 09:04:51.049257 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:04:51.049257 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:04:51.049257 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:04:51.049257 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:04:51.049257 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:04:51.049257 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:04:51.049257 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:04:51.049257 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:04:51.050108 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 09:04:51.050108 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 09:04:51.050108 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 09:04:51.050108 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 09:04:51.050108 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 09:04:51.050108 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 09:04:51.050108 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 09:04:51.050108 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 14 09:04:51.054802 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 256/522] 10.222.0.26 () {56 vars in 1111 bytes} [Thu Jul 14 09:04:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:04:51.055735 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d7e934-286a-412f-9619-b75d259fec38 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:51.144189 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8f93d7-39ee-4765-a35d-184c95c5cb0d None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:51.160653 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d7e934-286a-412f-9619-b75d259fec38 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:51.171859 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:51.181300 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37a697f-c1ea-4bbf-b44c-1e37844f7572 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:51.181968 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37a697f-c1ea-4bbf-b44c-1e37844f7572 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:51.182247 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37a697f-c1ea-4bbf-b44c-1e37844f7572 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:51.182578 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37a697f-c1ea-4bbf-b44c-1e37844f7572 None admin] PATH_INFO: `/v3/users` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:51.183388 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37a697f-c1ea-4bbf-b44c-1e37844f7572 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:04:51.183695 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37a697f-c1ea-4bbf-b44c-1e37844f7572 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:51.185205 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37a697f-c1ea-4bbf-b44c-1e37844f7572 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:51.197410 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8f7168-e466-4535-830d-96125a95a15d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:51.198080 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8f7168-e466-4535-830d-96125a95a15d None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:51.198366 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8f7168-e466-4535-830d-96125a95a15d None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:51.198644 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8f7168-e466-4535-830d-96125a95a15d None admin] PATH_INFO: `/v3/domains` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:51.199408 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8f7168-e466-4535-830d-96125a95a15d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:04:51.200026 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8f7168-e466-4535-830d-96125a95a15d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:51.201168 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8f7168-e466-4535-830d-96125a95a15d None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:51.207752 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 267/523] 10.222.0.26 () {56 vars in 1139 bytes} [Thu Jul 14 09:04:51 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:04:51.207752 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37a697f-c1ea-4bbf-b44c-1e37844f7572 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:51.222312 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 257/524] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Jul 14 09:04:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:04:51.222312 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8f7168-e466-4535-830d-96125a95a15d None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:51.269591 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:51.311075 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37a697f-c1ea-4bbf-b44c-1e37844f7572 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:51.332409 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8f7168-e466-4535-830d-96125a95a15d None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:51.345719 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ff665f-5505-4e43-b8ed-712cf7ff4487 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:51.346371 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ff665f-5505-4e43-b8ed-712cf7ff4487 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:51.346657 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ff665f-5505-4e43-b8ed-712cf7ff4487 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:51.346927 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ff665f-5505-4e43-b8ed-712cf7ff4487 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:51.369490 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6204d492-a537-4e74-8a10-e22542a2189b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:51.370275 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6204d492-a537-4e74-8a10-e22542a2189b None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:51.370559 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6204d492-a537-4e74-8a10-e22542a2189b None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:51.370914 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6204d492-a537-4e74-8a10-e22542a2189b None admin] PATH_INFO: `/v3/projects` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:51.371729 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6204d492-a537-4e74-8a10-e22542a2189b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:04:51.372039 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6204d492-a537-4e74-8a10-e22542a2189b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:51.373799 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6204d492-a537-4e74-8a10-e22542a2189b None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:51.397739 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 258/525] 10.222.0.26 () {56 vars in 1148 bytes} [Thu Jul 14 09:04:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:04:51.398115 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:51.398610 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6204d492-a537-4e74-8a10-e22542a2189b None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:51.417034 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ff665f-5505-4e43-b8ed-712cf7ff4487 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:51.418789 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ff665f-5505-4e43-b8ed-712cf7ff4487 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:51.514531 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6204d492-a537-4e74-8a10-e22542a2189b None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:51.517138 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:51.521929 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 268/526] 10.222.0.26 () {58 vars in 1282 bytes} [Thu Jul 14 09:04:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 09:04:51.522770 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ff665f-5505-4e43-b8ed-712cf7ff4487 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:51.559701 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d86831a-3a5b-45f5-a96c-83d0ac11fed9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:51.560492 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d86831a-3a5b-45f5-a96c-83d0ac11fed9 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:51.560840 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d86831a-3a5b-45f5-a96c-83d0ac11fed9 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:51.561119 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d86831a-3a5b-45f5-a96c-83d0ac11fed9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:51.617006 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:51.634032 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d86831a-3a5b-45f5-a96c-83d0ac11fed9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:51.634298 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ff665f-5505-4e43-b8ed-712cf7ff4487 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:51.635672 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d86831a-3a5b-45f5-a96c-83d0ac11fed9 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:51.668661 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b4eaeb-5074-44ae-a3d5-bd3f79fb6b01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:51.669484 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b4eaeb-5074-44ae-a3d5-bd3f79fb6b01 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:51.669776 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b4eaeb-5074-44ae-a3d5-bd3f79fb6b01 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:51.670067 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b4eaeb-5074-44ae-a3d5-bd3f79fb6b01 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:51.677031 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b4eaeb-5074-44ae-a3d5-bd3f79fb6b01 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:51.678601 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b4eaeb-5074-44ae-a3d5-bd3f79fb6b01 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:51.693207 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-43b4eaeb-5074-44ae-a3d5-bd3f79fb6b01 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 09:04:51.693207 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:04:51.693207 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:04:51.693207 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:04:51.693207 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:04:51.693207 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:04:51.693207 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:04:51.693207 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:04:51.693207 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:04:51.693207 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:04:51.693207 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:04:51.693207 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:04:51.693207 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 09:04:51.693207 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 09:04:51.693207 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 09:04:51.694173 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 09:04:51.694173 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:04:51.694173 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:04:51.694173 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:04:51.694173 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:04:51.694173 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:04:51.694173 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:04:51.694173 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:04:51.694173 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 14 09:04:51.694173 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:04:51.694173 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:04:51.694173 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:04:51.694173 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:04:51.694173 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:04:51.694173 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:04:51.694173 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:04:51.694173 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:04:51.695151 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 09:04:51.695151 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 09:04:51.695151 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 09:04:51.695151 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 09:04:51.695151 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 09:04:51.695151 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 09:04:51.695151 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 09:04:51.695151 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 14 09:04:51.698999 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 269/527] 10.222.0.26 () {56 vars in 1111 bytes} [Thu Jul 14 09:04:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:04:51.699819 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b4eaeb-5074-44ae-a3d5-bd3f79fb6b01 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:51.755767 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 259/528] 10.222.0.26 () {58 vars in 1282 bytes} [Thu Jul 14 09:04:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 09:04:51.755767 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d86831a-3a5b-45f5-a96c-83d0ac11fed9 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:51.756471 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:51.819080 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b4eaeb-5074-44ae-a3d5-bd3f79fb6b01 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:51.856405 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92827227-06d2-45b8-b102-6dc731be686d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:51.857104 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92827227-06d2-45b8-b102-6dc731be686d None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:51.857390 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92827227-06d2-45b8-b102-6dc731be686d None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:51.857656 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92827227-06d2-45b8-b102-6dc731be686d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:51.858496 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92827227-06d2-45b8-b102-6dc731be686d None admin] RBAC: Adding query filter params (role.id=f5c487bc605f45df82ef6367c80fae70, scope.project.id=33bf210d015847439655b9943d1b0261, user.id=538fff3c5a2b487eb44ae7c1a1198102) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:04:51.859205 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92827227-06d2-45b8-b102-6dc731be686d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:51.860282 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92827227-06d2-45b8-b102-6dc731be686d None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:51.861914 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d86831a-3a5b-45f5-a96c-83d0ac11fed9 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:51.875249 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 270/529] 10.222.0.26 () {56 vars in 1374 bytes} [Thu Jul 14 09:04:51 2022] GET /identity/v3/role_assignments?user.id=538fff3c5a2b487eb44ae7c1a1198102&scope.project.id=33bf210d015847439655b9943d1b0261&role.id=f5c487bc605f45df82ef6367c80fae70 => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:04:51.875249 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92827227-06d2-45b8-b102-6dc731be686d None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:51.887369 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:51.907470 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df361bf3-17ab-4bc4-ac5b-febc0f0d565b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:51.908285 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df361bf3-17ab-4bc4-ac5b-febc0f0d565b None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:51.908569 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df361bf3-17ab-4bc4-ac5b-febc0f0d565b None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:51.908841 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df361bf3-17ab-4bc4-ac5b-febc0f0d565b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:51.924013 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df361bf3-17ab-4bc4-ac5b-febc0f0d565b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:51.926515 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df361bf3-17ab-4bc4-ac5b-febc0f0d565b None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:51.948719 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-df361bf3-17ab-4bc4-ac5b-febc0f0d565b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 09:04:51.948719 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:04:51.948719 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:04:51.948719 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:04:51.948719 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:04:51.948719 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:04:51.948719 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:04:51.948719 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:04:51.948719 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:04:51.948719 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:04:51.948719 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:04:51.948719 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:04:51.948719 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 09:04:51.948719 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 09:04:51.948719 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 09:04:51.949663 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 09:04:51.949663 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:04:51.949663 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:04:51.949663 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:04:51.949663 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:04:51.949663 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:04:51.949663 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:04:51.949663 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:04:51.949663 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 14 09:04:51.949663 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:04:51.949663 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:04:51.949663 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:04:51.949663 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:04:51.949663 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:04:51.949663 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:04:51.949663 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:04:51.949663 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:04:51.950591 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 09:04:51.950591 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 09:04:51.950591 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 09:04:51.950591 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 14 09:04:51.950591 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 260/530] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jul 14 09:04:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:04:51.951349 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df361bf3-17ab-4bc4-ac5b-febc0f0d565b None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:52.018956 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:52.026910 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92827227-06d2-45b8-b102-6dc731be686d None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:52.069435 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26cc27f-c7ae-4540-a871-660c2d5df8c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:52.070335 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26cc27f-c7ae-4540-a871-660c2d5df8c6 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:52.070761 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26cc27f-c7ae-4540-a871-660c2d5df8c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:52.071176 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26cc27f-c7ae-4540-a871-660c2d5df8c6 None admin] PATH_INFO: `/v3/projects` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:52.071351 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df361bf3-17ab-4bc4-ac5b-febc0f0d565b None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:52.072295 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26cc27f-c7ae-4540-a871-660c2d5df8c6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:04:52.072746 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26cc27f-c7ae-4540-a871-660c2d5df8c6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:52.075573 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26cc27f-c7ae-4540-a871-660c2d5df8c6 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:52.099131 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 271/531] 10.222.0.26 () {56 vars in 1148 bytes} [Thu Jul 14 09:04:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:04:52.100556 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26cc27f-c7ae-4540-a871-660c2d5df8c6 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:52.110514 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302c278e-c5dc-42e7-a4d1-2cf3f640f9d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:52.111910 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302c278e-c5dc-42e7-a4d1-2cf3f640f9d3 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:52.112292 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302c278e-c5dc-42e7-a4d1-2cf3f640f9d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:52.112666 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302c278e-c5dc-42e7-a4d1-2cf3f640f9d3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:52.128664 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302c278e-c5dc-42e7-a4d1-2cf3f640f9d3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:52.130433 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302c278e-c5dc-42e7-a4d1-2cf3f640f9d3 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:52.148625 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-302c278e-c5dc-42e7-a4d1-2cf3f640f9d3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 09:04:52.148625 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:04:52.148625 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:04:52.148625 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:04:52.148625 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:04:52.148625 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:04:52.148625 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:04:52.148625 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:04:52.148625 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:04:52.148625 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:04:52.148625 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:04:52.148625 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:04:52.148625 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 09:04:52.148625 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 09:04:52.148625 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 09:04:52.149504 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 09:04:52.149504 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:04:52.149504 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:04:52.149504 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:04:52.149504 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:04:52.149504 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:04:52.149504 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:04:52.149504 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:04:52.149504 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 14 09:04:52.149504 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:04:52.149504 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:04:52.149504 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:04:52.149504 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:04:52.149504 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:04:52.149504 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:04:52.149504 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:04:52.149504 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:04:52.150366 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 09:04:52.150366 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 09:04:52.150366 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 09:04:52.150366 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 14 09:04:52.157072 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 261/532] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jul 14 09:04:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:04:52.158127 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302c278e-c5dc-42e7-a4d1-2cf3f640f9d3 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:52.175722 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:52.213066 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26cc27f-c7ae-4540-a871-660c2d5df8c6 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:52.255058 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9696b32a-2b6c-4d2c-a7e3-fc2a7c8907d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:52.256069 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9696b32a-2b6c-4d2c-a7e3-fc2a7c8907d2 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:52.256403 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9696b32a-2b6c-4d2c-a7e3-fc2a7c8907d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:52.256770 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9696b32a-2b6c-4d2c-a7e3-fc2a7c8907d2 None admin] PATH_INFO: `/v3/domains` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:52.257698 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9696b32a-2b6c-4d2c-a7e3-fc2a7c8907d2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:04:52.258101 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9696b32a-2b6c-4d2c-a7e3-fc2a7c8907d2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:52.259744 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9696b32a-2b6c-4d2c-a7e3-fc2a7c8907d2 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:52.263465 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302c278e-c5dc-42e7-a4d1-2cf3f640f9d3 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:52.272177 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 272/533] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Jul 14 09:04:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:04:52.273008 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:52.274115 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9696b32a-2b6c-4d2c-a7e3-fc2a7c8907d2 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:52.301224 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7421fd1b-fc7b-402b-8167-8ce07957ae30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:52.302155 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7421fd1b-fc7b-402b-8167-8ce07957ae30 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:52.302583 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7421fd1b-fc7b-402b-8167-8ce07957ae30 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:52.302998 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7421fd1b-fc7b-402b-8167-8ce07957ae30 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:52.304004 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7421fd1b-fc7b-402b-8167-8ce07957ae30 None admin] RBAC: Adding query filter params (role.id=f5c487bc605f45df82ef6367c80fae70, scope.project.id=33bf210d015847439655b9943d1b0261, user.id=872dbb91c9e54f13b1a31fc8242126df) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:04:52.304450 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7421fd1b-fc7b-402b-8167-8ce07957ae30 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:52.306043 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7421fd1b-fc7b-402b-8167-8ce07957ae30 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:52.312017 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 262/534] 10.222.0.26 () {56 vars in 1374 bytes} [Thu Jul 14 09:04:52 2022] GET /identity/v3/role_assignments?user.id=872dbb91c9e54f13b1a31fc8242126df&scope.project.id=33bf210d015847439655b9943d1b0261&role.id=f5c487bc605f45df82ef6367c80fae70 => generated 259 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:04:52.313083 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7421fd1b-fc7b-402b-8167-8ce07957ae30 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:52.391093 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9696b32a-2b6c-4d2c-a7e3-fc2a7c8907d2 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:52.421726 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:52.422302 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7421fd1b-fc7b-402b-8167-8ce07957ae30 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:52.431876 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75475a60-bbf9-4ed4-ad1f-8b667537ffb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:52.432570 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75475a60-bbf9-4ed4-ad1f-8b667537ffb4 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:52.432855 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75475a60-bbf9-4ed4-ad1f-8b667537ffb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:52.433129 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75475a60-bbf9-4ed4-ad1f-8b667537ffb4 None admin] PATH_INFO: `/v3/domains` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:52.434047 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75475a60-bbf9-4ed4-ad1f-8b667537ffb4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:04:52.434269 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75475a60-bbf9-4ed4-ad1f-8b667537ffb4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:52.436038 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75475a60-bbf9-4ed4-ad1f-8b667537ffb4 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:52.450500 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 273/535] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Jul 14 09:04:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:04:52.460365 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75475a60-bbf9-4ed4-ad1f-8b667537ffb4 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:52.495243 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3865d7ba-9591-478d-a717-094880d63032 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:52.495243 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3865d7ba-9591-478d-a717-094880d63032 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:52.495243 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3865d7ba-9591-478d-a717-094880d63032 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:52.495243 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3865d7ba-9591-478d-a717-094880d63032 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:52.510134 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3865d7ba-9591-478d-a717-094880d63032 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:52.511668 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3865d7ba-9591-478d-a717-094880d63032 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:52.537809 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-3865d7ba-9591-478d-a717-094880d63032 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 09:04:52.537809 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:04:52.537809 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:04:52.537809 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:04:52.537809 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:04:52.537809 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:04:52.537809 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:04:52.537809 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:04:52.537809 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:04:52.537809 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:04:52.537809 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:04:52.537809 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:04:52.537809 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 09:04:52.537809 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 09:04:52.537809 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 09:04:52.538725 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 09:04:52.538725 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:04:52.538725 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:04:52.538725 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:04:52.538725 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:04:52.538725 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:04:52.538725 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:04:52.538725 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:04:52.538725 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 14 09:04:52.538725 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:04:52.538725 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:04:52.538725 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:04:52.538725 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:04:52.538725 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:04:52.538725 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:04:52.538725 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:04:52.538725 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:04:52.539627 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 09:04:52.539627 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 09:04:52.539627 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 09:04:52.539627 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 09:04:52.539627 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 09:04:52.539627 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 09:04:52.539627 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 09:04:52.539627 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 14 09:04:52.552929 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 263/536] 10.222.0.26 () {56 vars in 1111 bytes} [Thu Jul 14 09:04:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:04:52.555652 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3865d7ba-9591-478d-a717-094880d63032 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:52.598457 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:52.616922 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75475a60-bbf9-4ed4-ad1f-8b667537ffb4 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:52.674917 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3865d7ba-9591-478d-a717-094880d63032 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:52.682690 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af399924-bac9-4472-8e80-793986928e3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:52.683540 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af399924-bac9-4472-8e80-793986928e3c None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:52.683823 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af399924-bac9-4472-8e80-793986928e3c None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:52.684097 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af399924-bac9-4472-8e80-793986928e3c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:52.700529 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af399924-bac9-4472-8e80-793986928e3c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:52.702399 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af399924-bac9-4472-8e80-793986928e3c None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:52.725368 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-af399924-bac9-4472-8e80-793986928e3c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 09:04:52.725368 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:04:52.725368 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:04:52.725368 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:04:52.725368 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:04:52.725368 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:04:52.725368 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:04:52.725368 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:04:52.725368 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:04:52.725368 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:04:52.725368 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:04:52.725368 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:04:52.725368 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 09:04:52.725368 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 09:04:52.725368 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 09:04:52.726291 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 09:04:52.726291 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:04:52.726291 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:04:52.726291 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:04:52.726291 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:04:52.726291 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:04:52.726291 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:04:52.726291 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:04:52.726291 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 14 09:04:52.726291 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:04:52.726291 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:04:52.726291 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:04:52.726291 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:04:52.726291 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:04:52.726291 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:04:52.726291 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:04:52.726291 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:04:52.727137 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 09:04:52.727137 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 09:04:52.727137 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 09:04:52.727137 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 09:04:52.727137 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 09:04:52.727137 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 09:04:52.727137 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 09:04:52.727137 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 14 09:04:52.729891 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 274/537] 10.222.0.26 () {56 vars in 1111 bytes} [Thu Jul 14 09:04:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:04:52.736747 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af399924-bac9-4472-8e80-793986928e3c None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:52.745776 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e0e6e4-5c9e-4545-a33c-e69c0b825d65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:52.749495 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e0e6e4-5c9e-4545-a33c-e69c0b825d65 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:52.750043 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e0e6e4-5c9e-4545-a33c-e69c0b825d65 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:52.750517 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e0e6e4-5c9e-4545-a33c-e69c0b825d65 None admin] PATH_INFO: `/v3/projects` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:52.751548 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e0e6e4-5c9e-4545-a33c-e69c0b825d65 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:04:52.753700 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e0e6e4-5c9e-4545-a33c-e69c0b825d65 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:52.755670 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e0e6e4-5c9e-4545-a33c-e69c0b825d65 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:52.780214 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 264/538] 10.222.0.26 () {56 vars in 1148 bytes} [Thu Jul 14 09:04:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:04:52.785896 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e0e6e4-5c9e-4545-a33c-e69c0b825d65 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:52.831214 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:52.886297 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:52.896290 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e0e6e4-5c9e-4545-a33c-e69c0b825d65 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:52.931914 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d9a284-89a3-4c0d-acd6-1a4eaed9e606 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:52.932949 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d9a284-89a3-4c0d-acd6-1a4eaed9e606 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:52.933447 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d9a284-89a3-4c0d-acd6-1a4eaed9e606 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:52.933913 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d9a284-89a3-4c0d-acd6-1a4eaed9e606 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:52.934991 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d9a284-89a3-4c0d-acd6-1a4eaed9e606 None admin] RBAC: Adding query filter params (role.id=f5c487bc605f45df82ef6367c80fae70, scope.project.id=33bf210d015847439655b9943d1b0261, user.id=e76f1952a5b146808844b9f2b7395035) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:04:52.935540 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d9a284-89a3-4c0d-acd6-1a4eaed9e606 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:52.937182 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d9a284-89a3-4c0d-acd6-1a4eaed9e606 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:52.943591 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 265/539] 10.222.0.26 () {56 vars in 1374 bytes} [Thu Jul 14 09:04:52 2022] GET /identity/v3/role_assignments?user.id=e76f1952a5b146808844b9f2b7395035&scope.project.id=33bf210d015847439655b9943d1b0261&role.id=f5c487bc605f45df82ef6367c80fae70 => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:04:52.958552 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af399924-bac9-4472-8e80-793986928e3c None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:52.985468 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f031543c-e72d-43f9-a3d7-b80b8ef574be None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:52.985807 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f031543c-e72d-43f9-a3d7-b80b8ef574be None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:52.986031 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f031543c-e72d-43f9-a3d7-b80b8ef574be None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:52.987884 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 266/540] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:04:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 09:04:52.993584 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4498ea68-0799-4089-9f02-b6e13d4f1030 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:52.993873 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4498ea68-0799-4089-9f02-b6e13d4f1030 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:52.994152 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4498ea68-0799-4089-9f02-b6e13d4f1030 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:53.014020 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-4498ea68-0799-4089-9f02-b6e13d4f1030 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:04:53.015517 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-4498ea68-0799-4089-9f02-b6e13d4f1030 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:04:53.020579 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac84a628-96ee-4295-926b-14543d9cb541 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:53.021589 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac84a628-96ee-4295-926b-14543d9cb541 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:53.022116 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac84a628-96ee-4295-926b-14543d9cb541 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:53.022588 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac84a628-96ee-4295-926b-14543d9cb541 None admin] PATH_INFO: `/v3/projects` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:53.023652 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac84a628-96ee-4295-926b-14543d9cb541 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:04:53.024178 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac84a628-96ee-4295-926b-14543d9cb541 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:53.025843 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac84a628-96ee-4295-926b-14543d9cb541 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:53.058924 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 275/541] 10.222.0.26 () {56 vars in 1148 bytes} [Thu Jul 14 09:04:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 09:04:53.064909 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac84a628-96ee-4295-926b-14543d9cb541 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:53.137217 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 267/542] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:04:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 09:04:53.166078 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4498ea68-0799-4089-9f02-b6e13d4f1030 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:53.210358 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac84a628-96ee-4295-926b-14543d9cb541 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:53.221104 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:53.263234 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:53.273812 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4e2d69-af4f-4f44-802a-e50794ea22e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:53.280038 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4e2d69-af4f-4f44-802a-e50794ea22e0 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:53.280317 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4e2d69-af4f-4f44-802a-e50794ea22e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:53.280599 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4e2d69-af4f-4f44-802a-e50794ea22e0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:53.281459 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4e2d69-af4f-4f44-802a-e50794ea22e0 None admin] RBAC: Adding query filter params (role.id=f5c487bc605f45df82ef6367c80fae70, scope.project.id=33bf210d015847439655b9943d1b0261, user.id=952911c492444ffea4b211a24b564117) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:04:53.281774 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4e2d69-af4f-4f44-802a-e50794ea22e0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:53.283037 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4498ea68-0799-4089-9f02-b6e13d4f1030 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:53.283269 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4e2d69-af4f-4f44-802a-e50794ea22e0 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:53.301551 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 276/543] 10.222.0.26 () {56 vars in 1374 bytes} [Thu Jul 14 09:04:53 2022] GET /identity/v3/role_assignments?user.id=952911c492444ffea4b211a24b564117&scope.project.id=33bf210d015847439655b9943d1b0261&role.id=f5c487bc605f45df82ef6367c80fae70 => generated 259 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 09:04:53.322471 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ce6bba-6f87-4015-b6a1-5cc9af8db289 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:53.323675 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ce6bba-6f87-4015-b6a1-5cc9af8db289 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:53.324161 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ce6bba-6f87-4015-b6a1-5cc9af8db289 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:53.324626 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ce6bba-6f87-4015-b6a1-5cc9af8db289 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:53.337137 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ce6bba-6f87-4015-b6a1-5cc9af8db289 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 09:04:53.345557 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-d1ce6bba-6f87-4015-b6a1-5cc9af8db289 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 09:04:53.345557 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:04:53.345557 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:04:53.345557 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:04:53.345557 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:04:53.345557 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:04:53.345557 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:04:53.345557 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:04:53.345557 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:04:53.345557 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:04:53.345557 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:04:53.345557 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:04:53.345557 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 09:04:53.345557 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 09:04:53.345557 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 09:04:53.345557 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 09:04:53.346445 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 09:04:53.346445 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 09:04:53.346445 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 09:04:53.346445 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 09:04:53.346445 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:04:53.346445 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:04:53.346445 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:04:53.346445 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:04:53.346445 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:04:53.346445 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:04:53.346445 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:04:53.346445 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 14 09:04:53.346445 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:04:53.346445 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:04:53.346445 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:04:53.346445 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:04:53.347269 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:04:53.347269 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:04:53.347269 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:04:53.347269 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:04:53.347269 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 09:04:53.347269 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 09:04:53.347269 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 09:04:53.347269 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 09:04:53.347269 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 09:04:53.347269 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 09:04:53.347269 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 09:04:53.347269 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 14 09:04:53.349510 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 268/544] 10.222.0.26 () {56 vars in 1102 bytes} [Thu Jul 14 09:04:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:04:53.360393 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4e2d69-af4f-4f44-802a-e50794ea22e0 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:53.507271 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4e2d69-af4f-4f44-802a-e50794ea22e0 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:53.569348 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0bec56-1f13-407d-87ee-d9e591c07378 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:53.570032 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0bec56-1f13-407d-87ee-d9e591c07378 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:53.570318 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0bec56-1f13-407d-87ee-d9e591c07378 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:53.571413 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0bec56-1f13-407d-87ee-d9e591c07378 None admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:53.571413 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0bec56-1f13-407d-87ee-d9e591c07378 None admin] RBAC: Adding query filter params (name=service) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:04:53.571695 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0bec56-1f13-407d-87ee-d9e591c07378 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:53.573092 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0bec56-1f13-407d-87ee-d9e591c07378 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:53.582487 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 277/545] 10.222.0.26 () {56 vars in 1103 bytes} [Thu Jul 14 09:04:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:04:53.588204 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ce6bba-6f87-4015-b6a1-5cc9af8db289 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:53.732960 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ce6bba-6f87-4015-b6a1-5cc9af8db289 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:53.784868 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93193e79-eb0e-4397-b18e-facd53999938 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:53.785601 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93193e79-eb0e-4397-b18e-facd53999938 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:53.785885 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93193e79-eb0e-4397-b18e-facd53999938 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:53.786170 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93193e79-eb0e-4397-b18e-facd53999938 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:53.799642 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93193e79-eb0e-4397-b18e-facd53999938 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:53.801163 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93193e79-eb0e-4397-b18e-facd53999938 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:53.809384 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:53.824201 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-93193e79-eb0e-4397-b18e-facd53999938 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 09:04:53.824201 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:04:53.824201 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:04:53.824201 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:04:53.824201 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:04:53.824201 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:04:53.824201 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:04:53.824201 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:04:53.824201 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:04:53.824201 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:04:53.824201 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:04:53.824201 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:04:53.824201 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 09:04:53.824201 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 09:04:53.824201 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 09:04:53.825112 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 09:04:53.825112 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:04:53.825112 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:04:53.825112 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:04:53.825112 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:04:53.825112 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:04:53.825112 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:04:53.825112 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:04:53.825112 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 14 09:04:53.825112 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:04:53.825112 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:04:53.825112 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:04:53.825112 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:04:53.825112 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:04:53.825112 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:04:53.825112 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:04:53.825112 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:04:53.825953 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 09:04:53.825953 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 09:04:53.825953 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 09:04:53.825953 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 14 09:04:53.825953 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 269/546] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jul 14 09:04:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:04:53.831554 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0bec56-1f13-407d-87ee-d9e591c07378 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:53.877955 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:53.977023 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0bec56-1f13-407d-87ee-d9e591c07378 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:54.032112 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bf6fbd-44c7-44fc-90be-153540e00805 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:54.032794 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bf6fbd-44c7-44fc-90be-153540e00805 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:54.033081 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bf6fbd-44c7-44fc-90be-153540e00805 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:54.033360 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bf6fbd-44c7-44fc-90be-153540e00805 None admin] PATH_INFO: `/v3/domains` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:54.034101 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bf6fbd-44c7-44fc-90be-153540e00805 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:04:54.034446 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bf6fbd-44c7-44fc-90be-153540e00805 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:54.038063 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bf6fbd-44c7-44fc-90be-153540e00805 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:54.053879 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 278/547] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Jul 14 09:04:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:04:54.059532 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93193e79-eb0e-4397-b18e-facd53999938 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:54.195739 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93193e79-eb0e-4397-b18e-facd53999938 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:54.252847 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2608fb43-f00a-4c6f-88b9-9d7f1cb4847b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:54.253654 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2608fb43-f00a-4c6f-88b9-9d7f1cb4847b None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:54.253935 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2608fb43-f00a-4c6f-88b9-9d7f1cb4847b None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:54.254205 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2608fb43-f00a-4c6f-88b9-9d7f1cb4847b None admin] PATH_INFO: `/v3/users/swift` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:54.268376 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2608fb43-f00a-4c6f-88b9-9d7f1cb4847b None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:54.269917 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2608fb43-f00a-4c6f-88b9-9d7f1cb4847b None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:54.297479 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-2608fb43-f00a-4c6f-88b9-9d7f1cb4847b None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 14 09:04:54.297479 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:04:54.297479 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:04:54.297479 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:04:54.297479 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:04:54.297479 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:04:54.297479 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:04:54.297479 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:04:54.297479 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:04:54.297479 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:04:54.297479 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:04:54.297479 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:04:54.297479 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 09:04:54.297479 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 09:04:54.297479 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 09:04:54.297479 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 09:04:54.298389 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:04:54.298389 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:04:54.298389 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 09:04:54.298389 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 09:04:54.298389 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 09:04:54.298389 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 09:04:54.298389 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:04:54.298389 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:04:54.298389 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:04:54.298389 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:04:54.298389 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:04:54.298389 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 14 09:04:54.298389 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:04:54.298389 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:04:54.298389 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:04:54.298389 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:04:54.298389 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:04:54.298389 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:04:54.299856 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:04:54.299856 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:04:54.299856 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 09:04:54.299856 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 09:04:54.299856 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 09:04:54.299856 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 09:04:54.299856 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 09:04:54.299856 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 09:04:54.299856 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 14 09:04:54.299856 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 14 09:04:54.301321 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 270/548] 10.222.0.26 () {56 vars in 1096 bytes} [Thu Jul 14 09:04:54 2022] GET /identity/v3/users/swift => generated 83 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:04:54.313497 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bf6fbd-44c7-44fc-90be-153540e00805 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:54.437568 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:54.444394 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bf6fbd-44c7-44fc-90be-153540e00805 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:54.459450 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871be930-a48a-483c-ae4e-9290b4d1db15 None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:54.459749 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871be930-a48a-483c-ae4e-9290b4d1db15 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:54.460011 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871be930-a48a-483c-ae4e-9290b4d1db15 None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:54.461614 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 271/549] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:04:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:04:54.467698 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea95d9a-0ba3-4a52-867d-208b9093a5ca None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:54.468237 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea95d9a-0ba3-4a52-867d-208b9093a5ca None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:54.468748 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea95d9a-0ba3-4a52-867d-208b9093a5ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:54.485468 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a955796d-9d03-46a0-ac98-2672692132ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:54.486200 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a955796d-9d03-46a0-ac98-2672692132ff None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:54.486487 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a955796d-9d03-46a0-ac98-2672692132ff None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:54.486808 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a955796d-9d03-46a0-ac98-2672692132ff None admin] PATH_INFO: `/v3/users` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:54.487698 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a955796d-9d03-46a0-ac98-2672692132ff None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:04:54.488010 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a955796d-9d03-46a0-ac98-2672692132ff None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:54.489464 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a955796d-9d03-46a0-ac98-2672692132ff None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:54.506801 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-0ea95d9a-0ba3-4a52-867d-208b9093a5ca None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:04:54.508357 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-0ea95d9a-0ba3-4a52-867d-208b9093a5ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:04:54.519212 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 279/550] 10.222.0.26 () {56 vars in 1135 bytes} [Thu Jul 14 09:04:54 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:04:54.525772 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a955796d-9d03-46a0-ac98-2672692132ff None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:54.611577 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:54.631911 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 272/551] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:04:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 09:04:54.656659 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea95d9a-0ba3-4a52-867d-208b9093a5ca None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:54.696959 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a955796d-9d03-46a0-ac98-2672692132ff None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:54.735810 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfaf95f5-eb56-411e-bcd8-c0df60334a31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:54.736706 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaf95f5-eb56-411e-bcd8-c0df60334a31 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:54.737251 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaf95f5-eb56-411e-bcd8-c0df60334a31 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:54.737665 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaf95f5-eb56-411e-bcd8-c0df60334a31 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:54.761868 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:54.784351 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea95d9a-0ba3-4a52-867d-208b9093a5ca None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:54.815959 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfaf95f5-eb56-411e-bcd8-c0df60334a31 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:54.817652 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfaf95f5-eb56-411e-bcd8-c0df60334a31 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:54.828721 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbb3c7f-ecf9-4299-a18a-000b78550480 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:54.829583 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbb3c7f-ecf9-4299-a18a-000b78550480 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:54.829875 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbb3c7f-ecf9-4299-a18a-000b78550480 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:54.830157 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbb3c7f-ecf9-4299-a18a-000b78550480 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:54.843945 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbb3c7f-ecf9-4299-a18a-000b78550480 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 09:04:54.855353 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-dcbb3c7f-ecf9-4299-a18a-000b78550480 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 09:04:54.855353 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:04:54.855353 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:04:54.855353 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:04:54.855353 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:04:54.855353 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:04:54.855353 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:04:54.855353 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:04:54.855353 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:04:54.855353 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:04:54.855353 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:04:54.855353 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:04:54.855353 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 09:04:54.855353 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 09:04:54.855353 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 09:04:54.855353 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 09:04:54.856339 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 09:04:54.856339 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 09:04:54.856339 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 09:04:54.856339 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 09:04:54.856339 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:04:54.856339 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:04:54.856339 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:04:54.856339 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:04:54.856339 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:04:54.856339 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:04:54.856339 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:04:54.856339 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 14 09:04:54.856339 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:04:54.856339 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:04:54.856339 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:04:54.856339 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:04:54.857312 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:04:54.857312 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:04:54.857312 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:04:54.857312 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:04:54.857312 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 09:04:54.857312 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 09:04:54.857312 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 09:04:54.857312 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 09:04:54.857312 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 09:04:54.857312 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 09:04:54.857312 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 09:04:54.857312 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 14 09:04:54.857312 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 273/552] 10.222.0.26 () {56 vars in 1102 bytes} [Thu Jul 14 09:04:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 09:04:54.858179 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aee869c-b18d-4ae9-b65e-e86653bee323 None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:54.858469 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aee869c-b18d-4ae9-b65e-e86653bee323 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:54.858754 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aee869c-b18d-4ae9-b65e-e86653bee323 None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:54.863267 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 274/553] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:04:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:04:54.864253 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aee869c-b18d-4ae9-b65e-e86653bee323 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:54.917332 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:54.947294 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 280/554] 10.222.0.26 () {58 vars in 1282 bytes} [Thu Jul 14 09:04:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 14 09:04:54.949624 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c15826-5fa9-400b-8e45-bc38cf7157d2 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:54.951630 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c15826-5fa9-400b-8e45-bc38cf7157d2 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:54.952002 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c15826-5fa9-400b-8e45-bc38cf7157d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:54.973562 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-f1c15826-5fa9-400b-8e45-bc38cf7157d2 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:04:54.975526 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-f1c15826-5fa9-400b-8e45-bc38cf7157d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:04:54.978086 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aee869c-b18d-4ae9-b65e-e86653bee323 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:55.013686 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6a4f9d-a7a0-47f9-bc00-a7fc147b5a2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:55.014695 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6a4f9d-a7a0-47f9-bc00-a7fc147b5a2a None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:55.015154 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6a4f9d-a7a0-47f9-bc00-a7fc147b5a2a None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:55.015639 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6a4f9d-a7a0-47f9-bc00-a7fc147b5a2a None admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:55.016566 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6a4f9d-a7a0-47f9-bc00-a7fc147b5a2a None admin] RBAC: Adding query filter params (name=service) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:04:55.017196 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6a4f9d-a7a0-47f9-bc00-a7fc147b5a2a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:55.018811 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6a4f9d-a7a0-47f9-bc00-a7fc147b5a2a None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:55.028387 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 275/555] 10.222.0.26 () {56 vars in 1103 bytes} [Thu Jul 14 09:04:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:04:55.029581 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6a4f9d-a7a0-47f9-bc00-a7fc147b5a2a None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:55.033664 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:55.059653 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 281/556] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:04:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 09:04:55.060859 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c15826-5fa9-400b-8e45-bc38cf7157d2 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:55.143106 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6a4f9d-a7a0-47f9-bc00-a7fc147b5a2a None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:55.154613 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:55.181111 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c15826-5fa9-400b-8e45-bc38cf7157d2 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:55.185743 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a95cd0-53a4-46bf-86cd-9a0a84d1f172 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:55.186553 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a95cd0-53a4-46bf-86cd-9a0a84d1f172 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:55.186840 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a95cd0-53a4-46bf-86cd-9a0a84d1f172 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:55.187117 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a95cd0-53a4-46bf-86cd-9a0a84d1f172 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:55.200345 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a95cd0-53a4-46bf-86cd-9a0a84d1f172 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:55.202098 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a95cd0-53a4-46bf-86cd-9a0a84d1f172 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:55.218239 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89033046-8aec-40da-947e-5bde2c81167e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:55.219388 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89033046-8aec-40da-947e-5bde2c81167e None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:55.219867 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89033046-8aec-40da-947e-5bde2c81167e None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:55.220347 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89033046-8aec-40da-947e-5bde2c81167e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:55.223529 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-90a95cd0-53a4-46bf-86cd-9a0a84d1f172 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 09:04:55.223529 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:04:55.223529 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:04:55.223529 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:04:55.223529 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:04:55.223529 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:04:55.223529 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:04:55.223529 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:04:55.223529 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:04:55.223529 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:04:55.223529 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:04:55.223529 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:04:55.223529 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 09:04:55.223529 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 09:04:55.223529 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 09:04:55.224390 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 09:04:55.224390 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:04:55.224390 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:04:55.224390 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:04:55.224390 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:04:55.224390 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:04:55.224390 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:04:55.224390 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:04:55.224390 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 14 09:04:55.224390 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:04:55.224390 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:04:55.224390 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:04:55.224390 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:04:55.224390 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:04:55.224390 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:04:55.224390 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:04:55.224390 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:04:55.225251 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 09:04:55.225251 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 09:04:55.225251 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 09:04:55.225251 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 14 09:04:55.229643 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 276/557] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jul 14 09:04:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:04:55.230998 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a95cd0-53a4-46bf-86cd-9a0a84d1f172 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:55.234525 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89033046-8aec-40da-947e-5bde2c81167e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:55.236438 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89033046-8aec-40da-947e-5bde2c81167e None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:55.256115 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-89033046-8aec-40da-947e-5bde2c81167e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 09:04:55.256115 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:04:55.256115 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:04:55.256115 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:04:55.256115 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:04:55.256115 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:04:55.256115 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:04:55.256115 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:04:55.256115 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:04:55.256115 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:04:55.256115 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:04:55.256115 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:04:55.256115 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 09:04:55.256115 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 09:04:55.256115 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 09:04:55.257889 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 09:04:55.257889 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:04:55.257889 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:04:55.257889 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:04:55.257889 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:04:55.257889 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:04:55.257889 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:04:55.257889 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:04:55.257889 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 14 09:04:55.257889 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:04:55.257889 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:04:55.257889 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:04:55.257889 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:04:55.257889 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:04:55.257889 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:04:55.257889 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:04:55.257889 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:04:55.258971 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 09:04:55.258971 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 09:04:55.258971 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 09:04:55.258971 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 14 09:04:55.258971 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 282/558] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jul 14 09:04:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:04:55.258971 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89033046-8aec-40da-947e-5bde2c81167e None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:55.294230 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:55.340214 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a95cd0-53a4-46bf-86cd-9a0a84d1f172 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:55.359671 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89033046-8aec-40da-947e-5bde2c81167e None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:55.377496 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d646780b-19f7-477b-abc7-7433c07b4c65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:55.378294 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d646780b-19f7-477b-abc7-7433c07b4c65 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:55.378577 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d646780b-19f7-477b-abc7-7433c07b4c65 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:55.378851 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d646780b-19f7-477b-abc7-7433c07b4c65 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:55.390908 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d646780b-19f7-477b-abc7-7433c07b4c65 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 09:04:55.395399 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3f810c-e3a1-4ba2-a1f5-239265795278 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:55.396080 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3f810c-e3a1-4ba2-a1f5-239265795278 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:55.396429 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3f810c-e3a1-4ba2-a1f5-239265795278 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:55.396769 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3f810c-e3a1-4ba2-a1f5-239265795278 None admin] PATH_INFO: `/v3/domains` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:55.397573 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3f810c-e3a1-4ba2-a1f5-239265795278 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:04:55.397959 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3f810c-e3a1-4ba2-a1f5-239265795278 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:55.398601 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-d646780b-19f7-477b-abc7-7433c07b4c65 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 09:04:55.398601 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:04:55.398601 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:04:55.398601 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:04:55.398601 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:04:55.398601 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:04:55.398601 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:04:55.398601 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:04:55.398601 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:04:55.398601 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:04:55.398601 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:04:55.398601 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:04:55.398601 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 09:04:55.398601 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 09:04:55.398601 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 09:04:55.398601 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 09:04:55.399744 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 09:04:55.399744 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 09:04:55.399744 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 09:04:55.399744 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 09:04:55.399744 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:04:55.399744 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:04:55.399744 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:04:55.399744 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:04:55.399744 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:04:55.399744 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:04:55.399744 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:04:55.399744 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 14 09:04:55.399744 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:04:55.399744 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:04:55.399744 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:04:55.399744 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:04:55.400689 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:04:55.400689 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:04:55.400689 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:04:55.400689 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:04:55.400689 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 09:04:55.400689 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 09:04:55.400689 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 09:04:55.400689 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 09:04:55.400689 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 09:04:55.400689 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 09:04:55.400689 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 09:04:55.400689 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 14 09:04:55.401333 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3f810c-e3a1-4ba2-a1f5-239265795278 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:55.401405 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 277/559] 10.222.0.26 () {56 vars in 1102 bytes} [Thu Jul 14 09:04:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:04:55.401405 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8cfd7d-3028-4ae6-94b1-74e99b17eb03 None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:55.401405 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8cfd7d-3028-4ae6-94b1-74e99b17eb03 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:55.401599 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8cfd7d-3028-4ae6-94b1-74e99b17eb03 None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:55.406944 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 278/560] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:04:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:04:55.406944 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8cfd7d-3028-4ae6-94b1-74e99b17eb03 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:55.423697 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 283/561] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Jul 14 09:04:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:04:55.423697 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03f382f-9b3c-4124-a1ff-aaef156312fe None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:55.423697 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03f382f-9b3c-4124-a1ff-aaef156312fe None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:55.423697 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03f382f-9b3c-4124-a1ff-aaef156312fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:55.432849 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:55.440374 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-f03f382f-9b3c-4124-a1ff-aaef156312fe None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:04:55.441663 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-f03f382f-9b3c-4124-a1ff-aaef156312fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:04:55.513944 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 284/562] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:04:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 09:04:55.514243 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8cfd7d-3028-4ae6-94b1-74e99b17eb03 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:55.514955 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03f382f-9b3c-4124-a1ff-aaef156312fe None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:55.542181 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:55.556396 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d369a1-6e4e-4a3c-9909-37eb5e36ccbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:55.557117 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d369a1-6e4e-4a3c-9909-37eb5e36ccbd None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:55.557397 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d369a1-6e4e-4a3c-9909-37eb5e36ccbd None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:55.557671 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d369a1-6e4e-4a3c-9909-37eb5e36ccbd None admin] PATH_INFO: `/v3/domains` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:55.558395 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d369a1-6e4e-4a3c-9909-37eb5e36ccbd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:04:55.558756 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d369a1-6e4e-4a3c-9909-37eb5e36ccbd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:55.560182 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d369a1-6e4e-4a3c-9909-37eb5e36ccbd None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:55.572349 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 279/563] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Jul 14 09:04:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:04:55.577308 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d369a1-6e4e-4a3c-9909-37eb5e36ccbd None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:55.628692 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03f382f-9b3c-4124-a1ff-aaef156312fe None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:55.660360 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:55.670645 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e86998-af8f-4e40-9a6f-d15e286105cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:55.671361 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e86998-af8f-4e40-9a6f-d15e286105cd None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:55.671647 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e86998-af8f-4e40-9a6f-d15e286105cd None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:55.671947 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e86998-af8f-4e40-9a6f-d15e286105cd None admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:55.672703 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e86998-af8f-4e40-9a6f-d15e286105cd None admin] RBAC: Adding query filter params (name=service) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:04:55.673009 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e86998-af8f-4e40-9a6f-d15e286105cd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:55.674387 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e86998-af8f-4e40-9a6f-d15e286105cd None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:55.681390 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d369a1-6e4e-4a3c-9909-37eb5e36ccbd None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:55.687513 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 285/564] 10.222.0.26 () {56 vars in 1103 bytes} [Thu Jul 14 09:04:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:04:55.687513 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e86998-af8f-4e40-9a6f-d15e286105cd None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:55.726349 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646c96f5-ba8e-494d-bcfc-841cd57b1ddb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:55.727646 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646c96f5-ba8e-494d-bcfc-841cd57b1ddb None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:55.728406 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646c96f5-ba8e-494d-bcfc-841cd57b1ddb None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:55.728885 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646c96f5-ba8e-494d-bcfc-841cd57b1ddb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:55.737684 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646c96f5-ba8e-494d-bcfc-841cd57b1ddb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:55.739650 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646c96f5-ba8e-494d-bcfc-841cd57b1ddb None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:55.760290 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-646c96f5-ba8e-494d-bcfc-841cd57b1ddb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 09:04:55.760290 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:04:55.760290 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:04:55.760290 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:04:55.760290 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:04:55.760290 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:04:55.760290 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:04:55.760290 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:04:55.760290 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:04:55.760290 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:04:55.760290 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:04:55.760290 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:04:55.760290 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 09:04:55.760290 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 09:04:55.760290 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 09:04:55.761233 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 09:04:55.761233 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:04:55.761233 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:04:55.761233 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:04:55.761233 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:04:55.761233 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:04:55.761233 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:04:55.761233 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:04:55.761233 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 14 09:04:55.761233 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:04:55.761233 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:04:55.761233 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:04:55.761233 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:04:55.761233 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:04:55.761233 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:04:55.761233 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:04:55.761233 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:04:55.762203 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 09:04:55.762203 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 09:04:55.762203 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 09:04:55.762203 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 09:04:55.762203 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 09:04:55.762203 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 09:04:55.762203 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 09:04:55.762203 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 14 09:04:55.762203 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 280/565] 10.222.0.26 () {56 vars in 1111 bytes} [Thu Jul 14 09:04:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:04:55.763540 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646c96f5-ba8e-494d-bcfc-841cd57b1ddb None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:55.822014 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:55.840498 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e86998-af8f-4e40-9a6f-d15e286105cd None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:55.876605 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646c96f5-ba8e-494d-bcfc-841cd57b1ddb None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:55.877873 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3cefb0-2357-4cad-bd90-fec09cf64268 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:55.879964 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3cefb0-2357-4cad-bd90-fec09cf64268 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:55.880265 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3cefb0-2357-4cad-bd90-fec09cf64268 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:55.880543 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3cefb0-2357-4cad-bd90-fec09cf64268 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:55.892870 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3cefb0-2357-4cad-bd90-fec09cf64268 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 09:04:55.901077 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-ab3cefb0-2357-4cad-bd90-fec09cf64268 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 09:04:55.901077 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:04:55.901077 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:04:55.901077 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:04:55.901077 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:04:55.901077 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:04:55.901077 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:04:55.901077 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:04:55.901077 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:04:55.901077 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:04:55.901077 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:04:55.901077 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:04:55.901077 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 09:04:55.901077 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 09:04:55.901077 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 09:04:55.901077 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 09:04:55.902032 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 09:04:55.902032 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 09:04:55.902032 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 09:04:55.902032 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 09:04:55.902032 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:04:55.902032 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:04:55.902032 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:04:55.902032 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:04:55.902032 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:04:55.902032 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:04:55.902032 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:04:55.902032 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 14 09:04:55.902032 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:04:55.902032 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:04:55.902032 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:04:55.902032 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:04:55.903680 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:04:55.903680 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:04:55.903680 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:04:55.903680 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:04:55.903680 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 09:04:55.903680 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 09:04:55.903680 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 09:04:55.903680 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 09:04:55.903680 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 09:04:55.903680 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 09:04:55.903680 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 09:04:55.903680 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 14 09:04:55.903680 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 286/566] 10.222.0.26 () {56 vars in 1102 bytes} [Thu Jul 14 09:04:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:04:55.903680 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3cefb0-2357-4cad-bd90-fec09cf64268 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:55.915004 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8557b176-38bf-4f9c-b61d-192973e45aaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:55.915780 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8557b176-38bf-4f9c-b61d-192973e45aaf None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:55.916068 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8557b176-38bf-4f9c-b61d-192973e45aaf None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:55.916340 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8557b176-38bf-4f9c-b61d-192973e45aaf None admin] PATH_INFO: `/v3/users/nova` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:55.924527 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8557b176-38bf-4f9c-b61d-192973e45aaf None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:55.925961 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8557b176-38bf-4f9c-b61d-192973e45aaf None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:55.942633 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-8557b176-38bf-4f9c-b61d-192973e45aaf None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 14 09:04:55.942633 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:04:55.942633 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:04:55.942633 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:04:55.942633 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:04:55.942633 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:04:55.942633 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:04:55.942633 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:04:55.942633 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:04:55.942633 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:04:55.942633 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:04:55.942633 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:04:55.942633 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 09:04:55.942633 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 09:04:55.942633 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 09:04:55.942633 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 09:04:55.943825 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:04:55.943825 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:04:55.943825 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 09:04:55.943825 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 09:04:55.943825 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 09:04:55.943825 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 09:04:55.943825 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:04:55.943825 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:04:55.943825 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:04:55.943825 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:04:55.943825 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:04:55.943825 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 14 09:04:55.943825 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:04:55.943825 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:04:55.943825 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:04:55.943825 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:04:55.943825 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:04:55.943825 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:04:55.944967 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:04:55.944967 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:04:55.944967 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 09:04:55.944967 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 09:04:55.944967 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 09:04:55.944967 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 09:04:55.944967 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 09:04:55.944967 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 09:04:55.944967 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 14 09:04:55.944967 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 14 09:04:55.944967 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 281/567] 10.222.0.26 () {56 vars in 1093 bytes} [Thu Jul 14 09:04:55 2022] GET /identity/v3/users/nova => generated 82 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:04:55.945696 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8557b176-38bf-4f9c-b61d-192973e45aaf None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:55.948639 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:56.012498 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3cefb0-2357-4cad-bd90-fec09cf64268 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:56.049720 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f049e472-2aa3-469e-8f0f-ad74aae2fb06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:56.050518 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f049e472-2aa3-469e-8f0f-ad74aae2fb06 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:56.050798 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f049e472-2aa3-469e-8f0f-ad74aae2fb06 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:56.051071 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f049e472-2aa3-469e-8f0f-ad74aae2fb06 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:56.054980 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8557b176-38bf-4f9c-b61d-192973e45aaf None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:56.061657 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f049e472-2aa3-469e-8f0f-ad74aae2fb06 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:56.063121 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f049e472-2aa3-469e-8f0f-ad74aae2fb06 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:56.069831 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:56.081427 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-f049e472-2aa3-469e-8f0f-ad74aae2fb06 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 09:04:56.081427 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:04:56.081427 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:04:56.081427 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:04:56.081427 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:04:56.081427 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:04:56.081427 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:04:56.081427 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:04:56.081427 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:04:56.081427 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:04:56.081427 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:04:56.081427 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:04:56.081427 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 09:04:56.081427 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 09:04:56.081427 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 09:04:56.082416 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 09:04:56.082416 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:04:56.082416 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:04:56.082416 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:04:56.082416 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:04:56.082416 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:04:56.082416 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:04:56.082416 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:04:56.082416 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 14 09:04:56.082416 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:04:56.082416 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:04:56.082416 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:04:56.082416 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:04:56.082416 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:04:56.082416 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:04:56.082416 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:04:56.082416 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:04:56.083416 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 09:04:56.083416 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 09:04:56.083416 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 09:04:56.083416 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 14 09:04:56.083416 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 287/568] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jul 14 09:04:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:04:56.084276 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38becad-974a-48f0-a128-d9438eab843d None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:56.084558 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38becad-974a-48f0-a128-d9438eab843d None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:56.084831 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38becad-974a-48f0-a128-d9438eab843d None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:56.086088 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 288/569] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:04:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:04:56.086761 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38becad-974a-48f0-a128-d9438eab843d None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:56.097382 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38cdd44-6c36-4473-9174-371b02c96526 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:56.098045 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38cdd44-6c36-4473-9174-371b02c96526 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:56.098328 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38cdd44-6c36-4473-9174-371b02c96526 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:56.098602 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38cdd44-6c36-4473-9174-371b02c96526 None admin] PATH_INFO: `/v3/projects` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:56.099424 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38cdd44-6c36-4473-9174-371b02c96526 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:04:56.099731 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38cdd44-6c36-4473-9174-371b02c96526 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:56.101176 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38cdd44-6c36-4473-9174-371b02c96526 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:56.112998 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 282/570] 10.222.0.26 () {56 vars in 1148 bytes} [Thu Jul 14 09:04:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:04:56.113807 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38cdd44-6c36-4473-9174-371b02c96526 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:56.176079 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:56.188910 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38becad-974a-48f0-a128-d9438eab843d None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:56.214724 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38cdd44-6c36-4473-9174-371b02c96526 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:56.222618 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838c7c97-6175-4e27-abab-69fe4ef23a2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:56.223308 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838c7c97-6175-4e27-abab-69fe4ef23a2b None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:56.223611 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838c7c97-6175-4e27-abab-69fe4ef23a2b None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:56.223869 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838c7c97-6175-4e27-abab-69fe4ef23a2b None admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:56.224622 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838c7c97-6175-4e27-abab-69fe4ef23a2b None admin] RBAC: Adding query filter params (name=service) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:04:56.224929 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838c7c97-6175-4e27-abab-69fe4ef23a2b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:56.226304 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838c7c97-6175-4e27-abab-69fe4ef23a2b None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:56.234173 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 289/571] 10.222.0.26 () {56 vars in 1103 bytes} [Thu Jul 14 09:04:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:04:56.234945 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838c7c97-6175-4e27-abab-69fe4ef23a2b None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:56.252001 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe1783a-ed30-42d8-b284-81b92b276adc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:56.252671 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe1783a-ed30-42d8-b284-81b92b276adc None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:56.252952 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe1783a-ed30-42d8-b284-81b92b276adc None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:56.253226 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe1783a-ed30-42d8-b284-81b92b276adc None admin] PATH_INFO: `/v3/users` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:56.254062 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe1783a-ed30-42d8-b284-81b92b276adc None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:04:56.254367 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe1783a-ed30-42d8-b284-81b92b276adc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:56.255833 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe1783a-ed30-42d8-b284-81b92b276adc None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:56.272807 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 283/572] 10.222.0.26 () {56 vars in 1133 bytes} [Thu Jul 14 09:04:56 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:04:56.273995 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de12cd7-a332-45b4-96e6-2b27db27f9ae None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:56.274279 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de12cd7-a332-45b4-96e6-2b27db27f9ae None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:56.274554 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de12cd7-a332-45b4-96e6-2b27db27f9ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:56.290330 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-8de12cd7-a332-45b4-96e6-2b27db27f9ae None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:04:56.291973 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-8de12cd7-a332-45b4-96e6-2b27db27f9ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:04:56.295524 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:56.341187 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838c7c97-6175-4e27-abab-69fe4ef23a2b None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:56.358593 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 284/573] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:04:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 09:04:56.359448 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de12cd7-a332-45b4-96e6-2b27db27f9ae None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:56.379248 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf700aa0-dcbb-4b3d-aa27-427cd23a5744 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:56.379981 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf700aa0-dcbb-4b3d-aa27-427cd23a5744 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:56.380262 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf700aa0-dcbb-4b3d-aa27-427cd23a5744 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:56.380539 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf700aa0-dcbb-4b3d-aa27-427cd23a5744 None admin] PATH_INFO: `/v3/domains` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:56.381271 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf700aa0-dcbb-4b3d-aa27-427cd23a5744 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:04:56.381576 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf700aa0-dcbb-4b3d-aa27-427cd23a5744 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:56.382953 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf700aa0-dcbb-4b3d-aa27-427cd23a5744 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:56.394750 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 290/574] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Jul 14 09:04:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:04:56.395565 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf700aa0-dcbb-4b3d-aa27-427cd23a5744 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:56.407664 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:56.456488 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de12cd7-a332-45b4-96e6-2b27db27f9ae None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:56.488651 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9464a3-1c87-4e06-bdde-27f69ed8439e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:56.489307 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9464a3-1c87-4e06-bdde-27f69ed8439e None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:56.489592 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9464a3-1c87-4e06-bdde-27f69ed8439e None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:56.489865 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9464a3-1c87-4e06-bdde-27f69ed8439e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:56.490697 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9464a3-1c87-4e06-bdde-27f69ed8439e None admin] RBAC: Adding query filter params (role.id=f5c487bc605f45df82ef6367c80fae70, scope.project.id=33bf210d015847439655b9943d1b0261, user.id=e5a43c80514b4a13af03c7f8ebcf7a5f) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:04:56.491004 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9464a3-1c87-4e06-bdde-27f69ed8439e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:56.492473 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9464a3-1c87-4e06-bdde-27f69ed8439e None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:56.493970 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf700aa0-dcbb-4b3d-aa27-427cd23a5744 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:56.497919 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 285/575] 10.222.0.26 () {56 vars in 1374 bytes} [Thu Jul 14 09:04:56 2022] GET /identity/v3/role_assignments?user.id=e5a43c80514b4a13af03c7f8ebcf7a5f&scope.project.id=33bf210d015847439655b9943d1b0261&role.id=f5c487bc605f45df82ef6367c80fae70 => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:04:56.498691 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9464a3-1c87-4e06-bdde-27f69ed8439e None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:56.515097 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:56.532508 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75927316-71ac-4f4f-bfa1-db27488931e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:56.533332 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75927316-71ac-4f4f-bfa1-db27488931e7 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:56.533620 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75927316-71ac-4f4f-bfa1-db27488931e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:56.533895 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75927316-71ac-4f4f-bfa1-db27488931e7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:56.546311 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75927316-71ac-4f4f-bfa1-db27488931e7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:56.547858 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75927316-71ac-4f4f-bfa1-db27488931e7 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:56.569121 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-75927316-71ac-4f4f-bfa1-db27488931e7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 09:04:56.569121 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:04:56.569121 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:04:56.569121 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:04:56.569121 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:04:56.569121 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:04:56.569121 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:04:56.569121 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:04:56.569121 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:04:56.569121 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:04:56.569121 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:04:56.569121 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:04:56.569121 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 09:04:56.569121 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 09:04:56.569121 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 09:04:56.570122 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 09:04:56.570122 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:04:56.570122 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:04:56.570122 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:04:56.570122 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:04:56.570122 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:04:56.570122 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:04:56.570122 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:04:56.570122 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 14 09:04:56.570122 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:04:56.570122 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:04:56.570122 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:04:56.570122 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:04:56.570122 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:04:56.570122 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:04:56.570122 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:04:56.570122 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:04:56.571016 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 09:04:56.571016 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 09:04:56.571016 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 09:04:56.571016 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 14 09:04:56.575131 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 291/576] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jul 14 09:04:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:04:56.576090 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75927316-71ac-4f4f-bfa1-db27488931e7 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:56.632974 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9464a3-1c87-4e06-bdde-27f69ed8439e None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:56.649302 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:56.674991 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ded5b52-7640-445b-ad9b-0181d12a3764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:56.675829 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ded5b52-7640-445b-ad9b-0181d12a3764 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:56.676118 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ded5b52-7640-445b-ad9b-0181d12a3764 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:56.676395 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ded5b52-7640-445b-ad9b-0181d12a3764 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:56.687801 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ded5b52-7640-445b-ad9b-0181d12a3764 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:56.689315 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ded5b52-7640-445b-ad9b-0181d12a3764 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:56.693160 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75927316-71ac-4f4f-bfa1-db27488931e7 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:56.709775 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-6ded5b52-7640-445b-ad9b-0181d12a3764 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 09:04:56.709775 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:04:56.709775 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:04:56.709775 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:04:56.709775 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:04:56.709775 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:04:56.709775 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:04:56.709775 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:04:56.709775 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:04:56.709775 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:04:56.709775 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:04:56.709775 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:04:56.709775 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 09:04:56.709775 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 09:04:56.709775 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 09:04:56.710713 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 09:04:56.710713 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:04:56.710713 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:04:56.710713 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:04:56.710713 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:04:56.710713 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:04:56.710713 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:04:56.710713 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:04:56.710713 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 14 09:04:56.710713 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:04:56.710713 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:04:56.710713 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:04:56.710713 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:04:56.710713 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:04:56.710713 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:04:56.710713 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:04:56.710713 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:04:56.712239 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 09:04:56.712239 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 09:04:56.712239 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 09:04:56.712239 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 14 09:04:56.712239 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 286/577] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jul 14 09:04:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:04:56.712239 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ded5b52-7640-445b-ad9b-0181d12a3764 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:56.732908 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65b9b97-8caf-41f0-995c-f386eb8b17dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:56.733718 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65b9b97-8caf-41f0-995c-f386eb8b17dd None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:56.734003 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65b9b97-8caf-41f0-995c-f386eb8b17dd None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:56.734276 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65b9b97-8caf-41f0-995c-f386eb8b17dd None admin] PATH_INFO: `/v3/roles/service` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:56.744891 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65b9b97-8caf-41f0-995c-f386eb8b17dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 09:04:56.752884 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-c65b9b97-8caf-41f0-995c-f386eb8b17dd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 09:04:56.752884 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:04:56.752884 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:04:56.752884 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:04:56.752884 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:04:56.752884 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:04:56.752884 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:04:56.752884 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:04:56.752884 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:04:56.752884 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:04:56.752884 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:04:56.752884 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:04:56.752884 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 09:04:56.752884 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 09:04:56.752884 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 09:04:56.752884 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 09:04:56.753851 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 09:04:56.753851 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 09:04:56.753851 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 09:04:56.753851 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 09:04:56.753851 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:04:56.753851 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:04:56.753851 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:04:56.753851 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:04:56.753851 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:04:56.753851 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:04:56.753851 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:04:56.753851 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 14 09:04:56.753851 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:04:56.753851 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:04:56.753851 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:04:56.753851 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:04:56.754714 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:04:56.754714 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:04:56.754714 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:04:56.754714 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:04:56.754714 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 09:04:56.754714 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 09:04:56.754714 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 09:04:56.754714 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 09:04:56.754714 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 09:04:56.754714 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 09:04:56.754714 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 09:04:56.754714 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 14 09:04:56.754714 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 292/578] 10.222.0.26 () {56 vars in 1102 bytes} [Thu Jul 14 09:04:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:04:56.755453 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65b9b97-8caf-41f0-995c-f386eb8b17dd None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:56.795588 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:56.831446 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ded5b52-7640-445b-ad9b-0181d12a3764 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:56.866532 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65b9b97-8caf-41f0-995c-f386eb8b17dd None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:56.873339 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbdb948-3e65-4f52-94bf-4d57e63d035b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:56.874531 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbdb948-3e65-4f52-94bf-4d57e63d035b None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:56.874811 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbdb948-3e65-4f52-94bf-4d57e63d035b None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:56.875081 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbdb948-3e65-4f52-94bf-4d57e63d035b None admin] PATH_INFO: `/v3/users/glance` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:56.884164 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbdb948-3e65-4f52-94bf-4d57e63d035b None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:56.885608 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbdb948-3e65-4f52-94bf-4d57e63d035b None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:56.900199 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-1cbdb948-3e65-4f52-94bf-4d57e63d035b None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 14 09:04:56.900199 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:04:56.900199 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:04:56.900199 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:04:56.900199 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:04:56.900199 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:04:56.900199 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:04:56.900199 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:04:56.900199 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:04:56.900199 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:04:56.900199 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:04:56.900199 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:04:56.900199 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 09:04:56.900199 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 09:04:56.900199 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 09:04:56.900199 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 09:04:56.901179 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:04:56.901179 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:04:56.901179 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 09:04:56.901179 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 09:04:56.901179 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 09:04:56.901179 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 09:04:56.901179 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:04:56.901179 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:04:56.901179 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:04:56.901179 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:04:56.901179 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:04:56.901179 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 14 09:04:56.901179 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:04:56.901179 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:04:56.901179 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:04:56.901179 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:04:56.901179 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:04:56.901179 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:04:56.902203 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:04:56.902203 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:04:56.902203 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 09:04:56.902203 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 09:04:56.902203 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 09:04:56.902203 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 09:04:56.902203 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 09:04:56.902203 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 09:04:56.902203 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 14 09:04:56.902203 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 14 09:04:56.902767 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a176be8c-c094-4779-a9df-b91198c6aebc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:56.902767 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a176be8c-c094-4779-a9df-b91198c6aebc None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:56.902767 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a176be8c-c094-4779-a9df-b91198c6aebc None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:56.902767 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a176be8c-c094-4779-a9df-b91198c6aebc None admin] PATH_INFO: `/v3/domains` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:56.903031 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a176be8c-c094-4779-a9df-b91198c6aebc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:04:56.903031 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a176be8c-c094-4779-a9df-b91198c6aebc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:56.904374 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a176be8c-c094-4779-a9df-b91198c6aebc None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:56.906049 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 287/579] 10.222.0.26 () {56 vars in 1099 bytes} [Thu Jul 14 09:04:56 2022] GET /identity/v3/users/glance => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:04:56.906730 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbdb948-3e65-4f52-94bf-4d57e63d035b None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:56.919647 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 293/580] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Jul 14 09:04:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:04:56.920598 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a176be8c-c094-4779-a9df-b91198c6aebc None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:56.939559 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:57.017875 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbdb948-3e65-4f52-94bf-4d57e63d035b None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:57.020978 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:57.031310 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a176be8c-c094-4779-a9df-b91198c6aebc None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:57.054014 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bd44ea-1e35-46e1-b633-8b921eac045d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:57.054665 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bd44ea-1e35-46e1-b633-8b921eac045d None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:57.054955 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bd44ea-1e35-46e1-b633-8b921eac045d None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:57.055251 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bd44ea-1e35-46e1-b633-8b921eac045d None admin] PATH_INFO: `/v3/domains` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:57.055972 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bd44ea-1e35-46e1-b633-8b921eac045d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:04:57.056276 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bd44ea-1e35-46e1-b633-8b921eac045d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:57.057649 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bd44ea-1e35-46e1-b633-8b921eac045d None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:57.074254 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 288/581] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Jul 14 09:04:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:04:57.074434 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa14c63-a223-4ee7-b5d7-35af781a470c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:57.074560 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bd44ea-1e35-46e1-b633-8b921eac045d None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:57.074652 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa14c63-a223-4ee7-b5d7-35af781a470c None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:57.074652 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa14c63-a223-4ee7-b5d7-35af781a470c None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:57.074652 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa14c63-a223-4ee7-b5d7-35af781a470c None admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:57.074652 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa14c63-a223-4ee7-b5d7-35af781a470c None admin] RBAC: Adding query filter params (name=service) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:04:57.074652 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa14c63-a223-4ee7-b5d7-35af781a470c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:57.074652 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa14c63-a223-4ee7-b5d7-35af781a470c None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:57.085375 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 294/582] 10.222.0.26 () {56 vars in 1103 bytes} [Thu Jul 14 09:04:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:04:57.086758 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa14c63-a223-4ee7-b5d7-35af781a470c None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:57.162152 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:57.178854 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bd44ea-1e35-46e1-b633-8b921eac045d None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:57.201029 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa14c63-a223-4ee7-b5d7-35af781a470c None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:57.213055 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d4b961-0e16-4e33-bd62-b2d7c1eecceb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:57.213729 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d4b961-0e16-4e33-bd62-b2d7c1eecceb None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:57.214013 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d4b961-0e16-4e33-bd62-b2d7c1eecceb None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:57.214291 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d4b961-0e16-4e33-bd62-b2d7c1eecceb None admin] PATH_INFO: `/v3/users` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:57.215961 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d4b961-0e16-4e33-bd62-b2d7c1eecceb None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:04:57.216273 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d4b961-0e16-4e33-bd62-b2d7c1eecceb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:57.217715 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d4b961-0e16-4e33-bd62-b2d7c1eecceb None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:57.239519 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 289/583] 10.222.0.26 () {56 vars in 1137 bytes} [Thu Jul 14 09:04:57 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:04:57.239519 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d4b961-0e16-4e33-bd62-b2d7c1eecceb None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:57.247739 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562cc904-9a79-4c79-8f3a-5e84d896ea7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:57.248611 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562cc904-9a79-4c79-8f3a-5e84d896ea7e None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:57.248946 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562cc904-9a79-4c79-8f3a-5e84d896ea7e None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:57.249278 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562cc904-9a79-4c79-8f3a-5e84d896ea7e None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:57.257453 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562cc904-9a79-4c79-8f3a-5e84d896ea7e None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:57.258980 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562cc904-9a79-4c79-8f3a-5e84d896ea7e None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:57.274719 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-562cc904-9a79-4c79-8f3a-5e84d896ea7e None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 14 09:04:57.274719 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:04:57.274719 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:04:57.274719 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:04:57.274719 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:04:57.274719 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:04:57.274719 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:04:57.274719 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:04:57.274719 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:04:57.274719 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:04:57.274719 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:04:57.274719 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:04:57.274719 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 09:04:57.274719 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 09:04:57.274719 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 09:04:57.274719 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 09:04:57.275919 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:04:57.275919 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:04:57.275919 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 09:04:57.275919 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 09:04:57.275919 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 09:04:57.275919 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 09:04:57.275919 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:04:57.275919 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:04:57.275919 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:04:57.275919 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:04:57.275919 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:04:57.275919 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 14 09:04:57.275919 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:04:57.275919 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:04:57.275919 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:04:57.275919 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:04:57.275919 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:04:57.275919 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:04:57.277082 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:04:57.277082 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:04:57.277082 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 09:04:57.277082 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 09:04:57.277082 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 09:04:57.277082 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 09:04:57.277082 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 09:04:57.277082 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 09:04:57.277082 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 14 09:04:57.277082 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 14 09:04:57.277742 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 295/584] 10.222.0.26 () {56 vars in 1099 bytes} [Thu Jul 14 09:04:57 2022] GET /identity/v3/users/cinder => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:04:57.282539 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562cc904-9a79-4c79-8f3a-5e84d896ea7e None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:57.300178 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:57.349160 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d4b961-0e16-4e33-bd62-b2d7c1eecceb None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:57.386751 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43531745-f866-42a0-901f-ca76f693af2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:57.387658 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43531745-f866-42a0-901f-ca76f693af2b None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:57.388009 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43531745-f866-42a0-901f-ca76f693af2b None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:57.388168 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562cc904-9a79-4c79-8f3a-5e84d896ea7e None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:57.388409 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43531745-f866-42a0-901f-ca76f693af2b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:57.396815 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43531745-f866-42a0-901f-ca76f693af2b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:57.399206 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43531745-f866-42a0-901f-ca76f693af2b None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:57.418834 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-43531745-f866-42a0-901f-ca76f693af2b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 09:04:57.418834 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:04:57.418834 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:04:57.418834 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:04:57.418834 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:04:57.418834 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:04:57.418834 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:04:57.418834 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:04:57.418834 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:04:57.418834 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:04:57.418834 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:04:57.418834 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:04:57.418834 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 09:04:57.418834 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 09:04:57.418834 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 09:04:57.419946 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 09:04:57.419946 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:04:57.419946 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:04:57.419946 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:04:57.419946 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:04:57.419946 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:04:57.419946 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:04:57.419946 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:04:57.419946 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 14 09:04:57.419946 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:04:57.419946 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:04:57.419946 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:04:57.419946 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:04:57.419946 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:04:57.419946 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:04:57.419946 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:04:57.419946 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:04:57.420904 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 09:04:57.420904 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 09:04:57.420904 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 09:04:57.420904 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 09:04:57.420904 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 09:04:57.420904 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 09:04:57.420904 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 09:04:57.420904 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 14 09:04:57.420904 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 290/585] 10.222.0.26 () {56 vars in 1111 bytes} [Thu Jul 14 09:04:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:04:57.421384 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43531745-f866-42a0-901f-ca76f693af2b None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:57.426138 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:57.426433 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f170d8-8bd9-4fca-96b1-08494d96b76c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:57.427323 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f170d8-8bd9-4fca-96b1-08494d96b76c None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:57.427725 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f170d8-8bd9-4fca-96b1-08494d96b76c None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:57.427885 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f170d8-8bd9-4fca-96b1-08494d96b76c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:57.440902 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f170d8-8bd9-4fca-96b1-08494d96b76c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:57.442406 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f170d8-8bd9-4fca-96b1-08494d96b76c None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:57.460474 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-84f170d8-8bd9-4fca-96b1-08494d96b76c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 09:04:57.460474 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:04:57.460474 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:04:57.460474 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:04:57.460474 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:04:57.460474 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:04:57.460474 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:04:57.460474 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:04:57.460474 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:04:57.460474 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:04:57.460474 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:04:57.460474 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:04:57.460474 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 09:04:57.460474 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 09:04:57.460474 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 09:04:57.461376 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 09:04:57.461376 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:04:57.461376 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:04:57.461376 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:04:57.461376 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:04:57.461376 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:04:57.461376 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:04:57.461376 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:04:57.461376 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 14 09:04:57.461376 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:04:57.461376 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:04:57.461376 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:04:57.461376 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:04:57.461376 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:04:57.461376 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:04:57.461376 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:04:57.461376 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:04:57.462264 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 09:04:57.462264 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 09:04:57.462264 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 09:04:57.462264 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 14 09:04:57.466430 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 296/586] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jul 14 09:04:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:04:57.467086 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f170d8-8bd9-4fca-96b1-08494d96b76c None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:57.525665 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43531745-f866-42a0-901f-ca76f693af2b None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:57.537750 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:57.564000 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b54274-8d7c-4a1a-850d-8d803812512d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:57.564914 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b54274-8d7c-4a1a-850d-8d803812512d None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:57.565265 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b54274-8d7c-4a1a-850d-8d803812512d None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:57.565608 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b54274-8d7c-4a1a-850d-8d803812512d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:57.577373 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b54274-8d7c-4a1a-850d-8d803812512d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:57.577697 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f170d8-8bd9-4fca-96b1-08494d96b76c None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:57.579016 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b54274-8d7c-4a1a-850d-8d803812512d None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:57.597069 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-21b54274-8d7c-4a1a-850d-8d803812512d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 09:04:57.597069 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:04:57.597069 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:04:57.597069 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:04:57.597069 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:04:57.597069 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:04:57.597069 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:04:57.597069 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:04:57.597069 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:04:57.597069 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:04:57.597069 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:04:57.597069 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:04:57.597069 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 09:04:57.597069 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 09:04:57.597069 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 09:04:57.598025 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 09:04:57.598025 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:04:57.598025 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:04:57.598025 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:04:57.598025 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:04:57.598025 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:04:57.598025 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:04:57.598025 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:04:57.598025 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 14 09:04:57.598025 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:04:57.598025 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:04:57.598025 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:04:57.598025 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:04:57.598025 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:04:57.598025 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:04:57.598025 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:04:57.598025 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:04:57.598914 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 09:04:57.598914 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 09:04:57.598914 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 09:04:57.598914 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 14 09:04:57.602825 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 291/587] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jul 14 09:04:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:04:57.603548 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b54274-8d7c-4a1a-850d-8d803812512d None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:57.614401 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd21ec1b-3397-4cfe-8234-716e05640e12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:57.615419 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd21ec1b-3397-4cfe-8234-716e05640e12 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:57.615993 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd21ec1b-3397-4cfe-8234-716e05640e12 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:57.616339 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd21ec1b-3397-4cfe-8234-716e05640e12 None admin] PATH_INFO: `/v3/users` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:57.617222 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd21ec1b-3397-4cfe-8234-716e05640e12 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:04:57.617635 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd21ec1b-3397-4cfe-8234-716e05640e12 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:57.619117 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd21ec1b-3397-4cfe-8234-716e05640e12 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:57.639694 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 297/588] 10.222.0.26 () {56 vars in 1137 bytes} [Thu Jul 14 09:04:57 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:04:57.640448 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd21ec1b-3397-4cfe-8234-716e05640e12 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:57.668361 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:57.718239 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b54274-8d7c-4a1a-850d-8d803812512d None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:57.749915 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd21ec1b-3397-4cfe-8234-716e05640e12 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:57.753598 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c425506-8356-4393-ba3a-a48e9851f242 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:57.754267 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c425506-8356-4393-ba3a-a48e9851f242 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:57.754551 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c425506-8356-4393-ba3a-a48e9851f242 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:57.754863 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c425506-8356-4393-ba3a-a48e9851f242 None admin] PATH_INFO: `/v3/projects` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:57.755649 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c425506-8356-4393-ba3a-a48e9851f242 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:04:57.755964 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c425506-8356-4393-ba3a-a48e9851f242 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:57.757411 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c425506-8356-4393-ba3a-a48e9851f242 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:57.770820 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 292/589] 10.222.0.26 () {56 vars in 1148 bytes} [Thu Jul 14 09:04:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:04:57.771697 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c425506-8356-4393-ba3a-a48e9851f242 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:57.774520 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:57.786303 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6aa2701-dee8-4379-8d78-57b133bf902c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:57.786971 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6aa2701-dee8-4379-8d78-57b133bf902c None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:57.787288 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6aa2701-dee8-4379-8d78-57b133bf902c None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:57.787565 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6aa2701-dee8-4379-8d78-57b133bf902c None admin] PATH_INFO: `/v3/domains` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:57.788300 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6aa2701-dee8-4379-8d78-57b133bf902c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:04:57.788613 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6aa2701-dee8-4379-8d78-57b133bf902c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:57.790012 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6aa2701-dee8-4379-8d78-57b133bf902c None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:57.802090 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 298/590] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Jul 14 09:04:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:04:57.802882 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6aa2701-dee8-4379-8d78-57b133bf902c None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:57.883875 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c425506-8356-4393-ba3a-a48e9851f242 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:57.900429 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:57.906167 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6aa2701-dee8-4379-8d78-57b133bf902c None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:57.921118 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b2825a-0103-4d31-9523-e39511919693 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:57.922088 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b2825a-0103-4d31-9523-e39511919693 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:57.922572 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b2825a-0103-4d31-9523-e39511919693 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:57.923051 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b2825a-0103-4d31-9523-e39511919693 None admin] PATH_INFO: `/v3/domains` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:57.923956 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b2825a-0103-4d31-9523-e39511919693 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:04:57.924408 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b2825a-0103-4d31-9523-e39511919693 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:57.925927 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b2825a-0103-4d31-9523-e39511919693 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:57.937927 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 293/591] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Jul 14 09:04:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:04:57.942957 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b2825a-0103-4d31-9523-e39511919693 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:57.943119 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d73357-eb15-4823-b28e-42b83139f28e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:57.944045 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d73357-eb15-4823-b28e-42b83139f28e None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:57.944412 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d73357-eb15-4823-b28e-42b83139f28e None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:57.944748 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d73357-eb15-4823-b28e-42b83139f28e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:57.959666 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d73357-eb15-4823-b28e-42b83139f28e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:57.961119 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d73357-eb15-4823-b28e-42b83139f28e None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:57.978396 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-d6d73357-eb15-4823-b28e-42b83139f28e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 09:04:57.978396 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:04:57.978396 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:04:57.978396 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:04:57.978396 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:04:57.978396 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:04:57.978396 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:04:57.978396 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:04:57.978396 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:04:57.978396 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:04:57.978396 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:04:57.978396 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:04:57.978396 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 09:04:57.978396 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 09:04:57.978396 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 09:04:57.979343 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 09:04:57.979343 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:04:57.979343 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:04:57.979343 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:04:57.979343 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:04:57.979343 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:04:57.979343 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:04:57.979343 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:04:57.979343 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 14 09:04:57.979343 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:04:57.979343 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:04:57.979343 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:04:57.979343 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:04:57.979343 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:04:57.979343 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:04:57.979343 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:04:57.979343 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:04:57.980329 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 09:04:57.980329 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 09:04:57.980329 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 09:04:57.980329 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 14 09:04:57.980329 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 299/592] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jul 14 09:04:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:04:57.980643 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d73357-eb15-4823-b28e-42b83139f28e None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:58.016535 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:58.060994 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b2825a-0103-4d31-9523-e39511919693 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:58.087400 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d73357-eb15-4823-b28e-42b83139f28e None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:58.095994 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582b519f-2691-4c1d-9d5b-d274ff15b4ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:58.096890 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582b519f-2691-4c1d-9d5b-d274ff15b4ab None admin] REQUEST_METHOD: `PUT` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:58.097343 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582b519f-2691-4c1d-9d5b-d274ff15b4ab None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:58.097540 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582b519f-2691-4c1d-9d5b-d274ff15b4ab None admin] PATH_INFO: `/v3/projects/33bf210d015847439655b9943d1b0261/users/538fff3c5a2b487eb44ae7c1a1198102/roles/f5c487bc605f45df82ef6367c80fae70` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:58.113408 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582b519f-2691-4c1d-9d5b-d274ff15b4ab None admin] RBAC: Authorizing `identity:create_grant(project_id=33bf210d015847439655b9943d1b0261, user_id=538fff3c5a2b487eb44ae7c1a1198102, role_id=f5c487bc605f45df82ef6367c80fae70)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:58.114860 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582b519f-2691-4c1d-9d5b-d274ff15b4ab None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:58.122736 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8014ae-9f62-48ee-a5a4-674fce1b1f76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:58.123559 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8014ae-9f62-48ee-a5a4-674fce1b1f76 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:58.123839 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8014ae-9f62-48ee-a5a4-674fce1b1f76 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:58.124109 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8014ae-9f62-48ee-a5a4-674fce1b1f76 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:58.139225 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 294/593] 10.222.0.26 () {58 vars in 1439 bytes} [Thu Jul 14 09:04:57 2022] PUT /identity/v3/projects/33bf210d015847439655b9943d1b0261/users/538fff3c5a2b487eb44ae7c1a1198102/roles/f5c487bc605f45df82ef6367c80fae70 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:04:58.139225 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582b519f-2691-4c1d-9d5b-d274ff15b4ab None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:58.141269 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8014ae-9f62-48ee-a5a4-674fce1b1f76 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:58.142875 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8014ae-9f62-48ee-a5a4-674fce1b1f76 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:58.160560 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-6a8014ae-9f62-48ee-a5a4-674fce1b1f76 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 14 09:04:58.160560 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:04:58.160560 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:04:58.160560 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:04:58.160560 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:04:58.160560 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:04:58.160560 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:04:58.160560 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:04:58.160560 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:04:58.160560 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:04:58.160560 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:04:58.160560 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:04:58.160560 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 09:04:58.160560 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 09:04:58.160560 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 09:04:58.160560 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 09:04:58.161631 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:04:58.161631 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:04:58.161631 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 09:04:58.161631 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 09:04:58.161631 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 09:04:58.161631 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 09:04:58.161631 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:04:58.161631 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:04:58.161631 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:04:58.161631 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:04:58.161631 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:04:58.161631 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 14 09:04:58.161631 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:04:58.161631 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:04:58.161631 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:04:58.161631 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:04:58.161631 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:04:58.161631 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:04:58.162590 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:04:58.162590 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:04:58.162590 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 09:04:58.162590 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 09:04:58.162590 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 09:04:58.162590 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 09:04:58.162590 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 09:04:58.162590 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 09:04:58.162590 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 14 09:04:58.162590 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 14 09:04:58.163142 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:58.167582 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 300/594] 10.222.0.26 () {56 vars in 1102 bytes} [Thu Jul 14 09:04:57 2022] GET /identity/v3/users/neutron => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:04:58.169442 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8014ae-9f62-48ee-a5a4-674fce1b1f76 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:58.249059 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582b519f-2691-4c1d-9d5b-d274ff15b4ab None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:58.288804 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd570a4-f788-4e68-962c-d3322131360b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:58.289610 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd570a4-f788-4e68-962c-d3322131360b None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:58.289890 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd570a4-f788-4e68-962c-d3322131360b None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:58.290154 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd570a4-f788-4e68-962c-d3322131360b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:58.290355 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:58.301706 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8014ae-9f62-48ee-a5a4-674fce1b1f76 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:58.302497 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd570a4-f788-4e68-962c-d3322131360b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:58.304010 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd570a4-f788-4e68-962c-d3322131360b None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:58.320584 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-3dd570a4-f788-4e68-962c-d3322131360b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 09:04:58.320584 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:04:58.320584 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:04:58.320584 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:04:58.320584 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:04:58.320584 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:04:58.320584 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:04:58.320584 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:04:58.320584 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:04:58.320584 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:04:58.320584 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:04:58.320584 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:04:58.320584 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 09:04:58.320584 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 09:04:58.320584 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 09:04:58.321680 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 09:04:58.321680 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:04:58.321680 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:04:58.321680 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:04:58.321680 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:04:58.321680 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:04:58.321680 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:04:58.321680 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:04:58.321680 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 14 09:04:58.321680 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:04:58.321680 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:04:58.321680 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:04:58.321680 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:04:58.321680 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:04:58.321680 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:04:58.321680 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:04:58.321680 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:04:58.322657 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 09:04:58.322657 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 09:04:58.322657 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 09:04:58.322657 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 09:04:58.322657 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 09:04:58.322657 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 09:04:58.322657 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 09:04:58.322657 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 14 09:04:58.322657 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 295/595] 10.222.0.26 () {56 vars in 1111 bytes} [Thu Jul 14 09:04:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:04:58.322657 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd570a4-f788-4e68-962c-d3322131360b None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:58.342249 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0b9516-04db-438e-a773-4a4f6eec52df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:58.343064 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0b9516-04db-438e-a773-4a4f6eec52df None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:58.343481 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0b9516-04db-438e-a773-4a4f6eec52df None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:58.343876 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0b9516-04db-438e-a773-4a4f6eec52df None admin] PATH_INFO: `/v3/domains` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:58.344718 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0b9516-04db-438e-a773-4a4f6eec52df None admin] RBAC: Adding query filter params (name=Default) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:04:58.345125 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0b9516-04db-438e-a773-4a4f6eec52df None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:58.346597 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0b9516-04db-438e-a773-4a4f6eec52df None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:58.359614 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 301/596] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Jul 14 09:04:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:04:58.360506 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0b9516-04db-438e-a773-4a4f6eec52df None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:58.426804 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:58.442638 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd570a4-f788-4e68-962c-d3322131360b None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:58.467655 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0b9516-04db-438e-a773-4a4f6eec52df None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:58.482797 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c3cf9d-70ba-4405-b0df-012559e7959d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:58.483719 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c3cf9d-70ba-4405-b0df-012559e7959d None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:58.484178 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c3cf9d-70ba-4405-b0df-012559e7959d None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:58.484557 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c3cf9d-70ba-4405-b0df-012559e7959d None admin] PATH_INFO: `/v3/users` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:58.485579 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c3cf9d-70ba-4405-b0df-012559e7959d None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:04:58.485994 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c3cf9d-70ba-4405-b0df-012559e7959d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:58.487697 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c3cf9d-70ba-4405-b0df-012559e7959d None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:58.506176 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 296/597] 10.222.0.26 () {56 vars in 1139 bytes} [Thu Jul 14 09:04:58 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:04:58.507096 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c3cf9d-70ba-4405-b0df-012559e7959d None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:58.511383 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dad233-8c82-4a39-842a-4ac636a95a3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:58.512112 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dad233-8c82-4a39-842a-4ac636a95a3d None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:58.512453 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dad233-8c82-4a39-842a-4ac636a95a3d None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:58.512806 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dad233-8c82-4a39-842a-4ac636a95a3d None admin] PATH_INFO: `/v3/projects` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:58.513738 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07dad233-8c82-4a39-842a-4ac636a95a3d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:04:58.514118 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07dad233-8c82-4a39-842a-4ac636a95a3d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:58.515637 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07dad233-8c82-4a39-842a-4ac636a95a3d None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:58.529587 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 302/598] 10.222.0.26 () {56 vars in 1148 bytes} [Thu Jul 14 09:04:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:04:58.532030 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59da506e-3337-4fda-af15-a2acdf2f1371 None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:58.532552 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59da506e-3337-4fda-af15-a2acdf2f1371 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:58.533014 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59da506e-3337-4fda-af15-a2acdf2f1371 None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:58.535070 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 303/599] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:04:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:04:58.535816 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59da506e-3337-4fda-af15-a2acdf2f1371 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:58.555341 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:58.627090 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c3cf9d-70ba-4405-b0df-012559e7959d None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:58.649957 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59da506e-3337-4fda-af15-a2acdf2f1371 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:58.662028 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abffb513-b3de-41fa-903f-7ca6a7625da2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:58.663102 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abffb513-b3de-41fa-903f-7ca6a7625da2 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:58.663422 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abffb513-b3de-41fa-903f-7ca6a7625da2 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:58.663703 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abffb513-b3de-41fa-903f-7ca6a7625da2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:58.671667 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abffb513-b3de-41fa-903f-7ca6a7625da2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:58.673113 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abffb513-b3de-41fa-903f-7ca6a7625da2 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:58.686962 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-abffb513-b3de-41fa-903f-7ca6a7625da2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 09:04:58.686962 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:04:58.686962 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:04:58.686962 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:04:58.686962 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:04:58.686962 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:04:58.686962 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:04:58.686962 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:04:58.686962 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:04:58.686962 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:04:58.686962 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:04:58.686962 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:04:58.686962 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 09:04:58.686962 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 09:04:58.686962 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 09:04:58.688052 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 09:04:58.688052 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:04:58.688052 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:04:58.688052 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:04:58.688052 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:04:58.688052 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:04:58.688052 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:04:58.688052 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:04:58.688052 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 14 09:04:58.688052 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:04:58.688052 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:04:58.688052 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:04:58.688052 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:04:58.688052 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:04:58.688052 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:04:58.688052 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:04:58.688052 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:04:58.689247 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 09:04:58.689247 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 09:04:58.689247 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 09:04:58.689247 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 09:04:58.689247 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 09:04:58.689247 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 09:04:58.689247 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 09:04:58.689247 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 14 09:04:58.691772 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3037b0ac-c814-4779-ad30-8d4b38b96ec6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:58.692780 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3037b0ac-c814-4779-ad30-8d4b38b96ec6 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:58.692889 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 297/600] 10.222.0.26 () {56 vars in 1111 bytes} [Thu Jul 14 09:04:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:04:58.693087 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3037b0ac-c814-4779-ad30-8d4b38b96ec6 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:58.693359 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3037b0ac-c814-4779-ad30-8d4b38b96ec6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:58.693703 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abffb513-b3de-41fa-903f-7ca6a7625da2 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:58.699507 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:58.704439 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3037b0ac-c814-4779-ad30-8d4b38b96ec6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:58.706848 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3037b0ac-c814-4779-ad30-8d4b38b96ec6 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:58.726885 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-3037b0ac-c814-4779-ad30-8d4b38b96ec6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 09:04:58.726885 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:04:58.726885 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:04:58.726885 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:04:58.726885 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:04:58.726885 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:04:58.726885 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:04:58.726885 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:04:58.726885 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:04:58.726885 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:04:58.726885 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:04:58.726885 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:04:58.726885 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 09:04:58.726885 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 09:04:58.726885 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 09:04:58.728081 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 09:04:58.728081 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:04:58.728081 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:04:58.728081 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:04:58.728081 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:04:58.728081 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:04:58.728081 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:04:58.728081 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:04:58.728081 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 14 09:04:58.728081 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:04:58.728081 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:04:58.728081 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:04:58.728081 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:04:58.728081 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:04:58.728081 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:04:58.728081 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:04:58.728081 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:04:58.729015 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 09:04:58.729015 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 09:04:58.729015 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 09:04:58.729015 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 14 09:04:58.732754 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 304/601] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jul 14 09:04:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:04:58.733974 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dd88f8-ac81-45dc-92c2-eb69c21a7ba9 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:58.734273 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dd88f8-ac81-45dc-92c2-eb69c21a7ba9 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:58.734540 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dd88f8-ac81-45dc-92c2-eb69c21a7ba9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:58.752023 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-10dd88f8-ac81-45dc-92c2-eb69c21a7ba9 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:04:58.753162 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-10dd88f8-ac81-45dc-92c2-eb69c21a7ba9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:04:58.806004 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abffb513-b3de-41fa-903f-7ca6a7625da2 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:58.818969 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:58.826049 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 305/602] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:04:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 09:04:58.827031 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10dd88f8-ac81-45dc-92c2-eb69c21a7ba9 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:58.846348 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b9b5ad-e005-4e37-a330-69c3cf4f883e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:58.847911 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b9b5ad-e005-4e37-a330-69c3cf4f883e None admin] REQUEST_METHOD: `PUT` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:58.848383 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b9b5ad-e005-4e37-a330-69c3cf4f883e None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:58.848664 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b9b5ad-e005-4e37-a330-69c3cf4f883e None admin] PATH_INFO: `/v3/projects/33bf210d015847439655b9943d1b0261/users/872dbb91c9e54f13b1a31fc8242126df/roles/f5c487bc605f45df82ef6367c80fae70` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:58.864835 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b9b5ad-e005-4e37-a330-69c3cf4f883e None admin] RBAC: Authorizing `identity:create_grant(project_id=33bf210d015847439655b9943d1b0261, user_id=872dbb91c9e54f13b1a31fc8242126df, role_id=f5c487bc605f45df82ef6367c80fae70)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:58.866281 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b9b5ad-e005-4e37-a330-69c3cf4f883e None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:58.891304 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 298/603] 10.222.0.26 () {58 vars in 1439 bytes} [Thu Jul 14 09:04:58 2022] PUT /identity/v3/projects/33bf210d015847439655b9943d1b0261/users/872dbb91c9e54f13b1a31fc8242126df/roles/f5c487bc605f45df82ef6367c80fae70 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:04:58.891304 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b9b5ad-e005-4e37-a330-69c3cf4f883e None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:58.952776 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:58.954665 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10dd88f8-ac81-45dc-92c2-eb69c21a7ba9 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:59.002203 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62277d7-f400-4adb-9c3d-1c973ff58843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:59.002878 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62277d7-f400-4adb-9c3d-1c973ff58843 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:59.003164 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62277d7-f400-4adb-9c3d-1c973ff58843 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:59.003470 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62277d7-f400-4adb-9c3d-1c973ff58843 None admin] PATH_INFO: `/v3/projects` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:59.004279 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62277d7-f400-4adb-9c3d-1c973ff58843 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:04:59.004589 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62277d7-f400-4adb-9c3d-1c973ff58843 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:59.006057 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62277d7-f400-4adb-9c3d-1c973ff58843 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:59.018212 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b9b5ad-e005-4e37-a330-69c3cf4f883e None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:59.019375 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 306/604] 10.222.0.26 () {56 vars in 1148 bytes} [Thu Jul 14 09:04:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:04:59.020290 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62277d7-f400-4adb-9c3d-1c973ff58843 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:59.067946 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bebd69d-6d36-4121-a8fe-ab161adc8f6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:59.067946 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bebd69d-6d36-4121-a8fe-ab161adc8f6d None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:59.067946 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bebd69d-6d36-4121-a8fe-ab161adc8f6d None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:59.067946 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bebd69d-6d36-4121-a8fe-ab161adc8f6d None admin] PATH_INFO: `/v3/domains` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:59.068999 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bebd69d-6d36-4121-a8fe-ab161adc8f6d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:04:59.069643 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bebd69d-6d36-4121-a8fe-ab161adc8f6d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:59.071703 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bebd69d-6d36-4121-a8fe-ab161adc8f6d None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:59.085965 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 299/605] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Jul 14 09:04:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:04:59.086953 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bebd69d-6d36-4121-a8fe-ab161adc8f6d None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:59.087769 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:59.132696 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62277d7-f400-4adb-9c3d-1c973ff58843 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:59.167690 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911f767b-a636-475b-ae13-7f659893285a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:59.168569 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911f767b-a636-475b-ae13-7f659893285a None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:59.169003 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911f767b-a636-475b-ae13-7f659893285a None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:59.169377 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911f767b-a636-475b-ae13-7f659893285a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:59.180453 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911f767b-a636-475b-ae13-7f659893285a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 09:04:59.188583 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-911f767b-a636-475b-ae13-7f659893285a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 09:04:59.188583 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:04:59.188583 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:04:59.188583 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:04:59.188583 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:04:59.188583 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:04:59.188583 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:04:59.188583 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:04:59.188583 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:04:59.188583 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:04:59.188583 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:04:59.188583 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:04:59.188583 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 09:04:59.188583 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 09:04:59.188583 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 09:04:59.188583 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 09:04:59.189774 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 09:04:59.189774 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 09:04:59.189774 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 09:04:59.189774 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 09:04:59.189774 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:04:59.189774 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:04:59.189774 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:04:59.189774 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:04:59.189774 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:04:59.189774 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:04:59.189774 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:04:59.189774 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 14 09:04:59.189774 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:04:59.189774 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:04:59.189774 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:04:59.189774 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:04:59.190811 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:04:59.190811 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:04:59.190811 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:04:59.190811 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:04:59.190811 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 09:04:59.190811 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 09:04:59.190811 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 09:04:59.190811 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 09:04:59.190811 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 09:04:59.190811 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 09:04:59.190811 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 09:04:59.190811 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 14 09:04:59.192861 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 307/606] 10.222.0.26 () {56 vars in 1102 bytes} [Thu Jul 14 09:04:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:04:59.198747 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911f767b-a636-475b-ae13-7f659893285a None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:59.203667 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bebd69d-6d36-4121-a8fe-ab161adc8f6d None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:59.243569 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:59.248249 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7c53b7-c899-4af1-8283-1628bfa17533 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:59.249430 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7c53b7-c899-4af1-8283-1628bfa17533 None admin] REQUEST_METHOD: `PUT` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:59.249719 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7c53b7-c899-4af1-8283-1628bfa17533 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:59.249998 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7c53b7-c899-4af1-8283-1628bfa17533 None admin] PATH_INFO: `/v3/projects/33bf210d015847439655b9943d1b0261/users/e76f1952a5b146808844b9f2b7395035/roles/f5c487bc605f45df82ef6367c80fae70` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:59.270480 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7c53b7-c899-4af1-8283-1628bfa17533 None admin] RBAC: Authorizing `identity:create_grant(project_id=33bf210d015847439655b9943d1b0261, user_id=e76f1952a5b146808844b9f2b7395035, role_id=f5c487bc605f45df82ef6367c80fae70)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:59.272230 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7c53b7-c899-4af1-8283-1628bfa17533 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:59.291062 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 300/607] 10.222.0.26 () {58 vars in 1439 bytes} [Thu Jul 14 09:04:59 2022] PUT /identity/v3/projects/33bf210d015847439655b9943d1b0261/users/e76f1952a5b146808844b9f2b7395035/roles/f5c487bc605f45df82ef6367c80fae70 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:04:59.294261 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7c53b7-c899-4af1-8283-1628bfa17533 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:59.325094 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911f767b-a636-475b-ae13-7f659893285a None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:59.360895 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:59.397974 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f4e177-047d-476e-96be-6c6f09f2f675 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:59.398913 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f4e177-047d-476e-96be-6c6f09f2f675 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:59.399222 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f4e177-047d-476e-96be-6c6f09f2f675 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:59.399501 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f4e177-047d-476e-96be-6c6f09f2f675 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:59.407719 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f4e177-047d-476e-96be-6c6f09f2f675 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:59.409235 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f4e177-047d-476e-96be-6c6f09f2f675 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:59.420611 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7c53b7-c899-4af1-8283-1628bfa17533 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:59.440966 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-17f4e177-047d-476e-96be-6c6f09f2f675 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 09:04:59.440966 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:04:59.440966 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:04:59.440966 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:04:59.440966 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:04:59.440966 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:04:59.440966 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:04:59.440966 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:04:59.440966 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:04:59.440966 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:04:59.440966 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:04:59.440966 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:04:59.440966 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 09:04:59.440966 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 09:04:59.440966 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 09:04:59.442107 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 09:04:59.442107 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:04:59.442107 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:04:59.442107 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:04:59.442107 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:04:59.442107 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:04:59.442107 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:04:59.442107 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:04:59.442107 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 14 09:04:59.442107 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:04:59.442107 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:04:59.442107 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:04:59.442107 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:04:59.442107 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:04:59.442107 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:04:59.442107 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:04:59.442107 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:04:59.443341 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 09:04:59.443341 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 09:04:59.443341 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 09:04:59.443341 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 09:04:59.443341 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 09:04:59.443341 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 09:04:59.443341 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 09:04:59.443341 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 14 09:04:59.448119 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 308/608] 10.222.0.26 () {56 vars in 1111 bytes} [Thu Jul 14 09:04:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:04:59.454132 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f4e177-047d-476e-96be-6c6f09f2f675 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:59.466658 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc59591-5ccc-4932-bd2f-db73e6a8a10a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:59.467370 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc59591-5ccc-4932-bd2f-db73e6a8a10a None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:59.467655 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc59591-5ccc-4932-bd2f-db73e6a8a10a None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:59.467929 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc59591-5ccc-4932-bd2f-db73e6a8a10a None admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:59.468748 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc59591-5ccc-4932-bd2f-db73e6a8a10a None admin] RBAC: Adding query filter params (name=service) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:04:59.469057 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc59591-5ccc-4932-bd2f-db73e6a8a10a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:59.470457 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc59591-5ccc-4932-bd2f-db73e6a8a10a None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:59.483240 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 301/609] 10.222.0.26 () {56 vars in 1103 bytes} [Thu Jul 14 09:04:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:04:59.484918 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc59591-5ccc-4932-bd2f-db73e6a8a10a None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:59.527589 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:59.615674 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc59591-5ccc-4932-bd2f-db73e6a8a10a None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:59.631335 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f4e177-047d-476e-96be-6c6f09f2f675 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:59.664069 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bca35bc-0f69-46b8-a708-99c3da440255 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:59.664871 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bca35bc-0f69-46b8-a708-99c3da440255 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:59.665154 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bca35bc-0f69-46b8-a708-99c3da440255 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:59.665411 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bca35bc-0f69-46b8-a708-99c3da440255 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:59.690280 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bca35bc-0f69-46b8-a708-99c3da440255 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:59.691680 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bca35bc-0f69-46b8-a708-99c3da440255 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:59.699601 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58f2d9a-66ad-46b9-a361-b4d47b085405 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:59.700269 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58f2d9a-66ad-46b9-a361-b4d47b085405 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:59.700548 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58f2d9a-66ad-46b9-a361-b4d47b085405 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:59.700819 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58f2d9a-66ad-46b9-a361-b4d47b085405 None admin] PATH_INFO: `/v3/projects` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:59.701612 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58f2d9a-66ad-46b9-a361-b4d47b085405 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:04:59.701915 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58f2d9a-66ad-46b9-a361-b4d47b085405 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:59.703362 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58f2d9a-66ad-46b9-a361-b4d47b085405 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:59.728621 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 309/610] 10.222.0.26 () {56 vars in 1148 bytes} [Thu Jul 14 09:04:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 09:04:59.730290 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-6bca35bc-0f69-46b8-a708-99c3da440255 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 09:04:59.730290 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:04:59.730290 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:04:59.730290 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:04:59.730290 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:04:59.730290 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:04:59.730290 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:04:59.730290 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:04:59.730290 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:04:59.730290 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:04:59.730290 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:04:59.730290 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:04:59.730290 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 09:04:59.730290 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 09:04:59.730290 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 09:04:59.731225 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 09:04:59.731225 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:04:59.731225 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:04:59.731225 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:04:59.731225 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:04:59.731225 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:04:59.731225 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:04:59.731225 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:04:59.731225 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 14 09:04:59.731225 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:04:59.731225 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:04:59.731225 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:04:59.731225 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:04:59.731225 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:04:59.731225 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:04:59.731225 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:04:59.731225 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:04:59.732522 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 09:04:59.732522 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 09:04:59.732522 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 09:04:59.732522 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 14 09:04:59.732522 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 302/611] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jul 14 09:04:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 09:04:59.734917 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58f2d9a-66ad-46b9-a361-b4d47b085405 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:59.751924 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bca35bc-0f69-46b8-a708-99c3da440255 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:04:59.781563 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:59.846578 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58f2d9a-66ad-46b9-a361-b4d47b085405 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:04:59.881911 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea17274f-2206-4d81-b4fe-c6ab2c0382e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:04:59.882807 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea17274f-2206-4d81-b4fe-c6ab2c0382e4 None admin] REQUEST_METHOD: `PUT` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:04:59.883215 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea17274f-2206-4d81-b4fe-c6ab2c0382e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:04:59.883484 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea17274f-2206-4d81-b4fe-c6ab2c0382e4 None admin] PATH_INFO: `/v3/projects/33bf210d015847439655b9943d1b0261/users/952911c492444ffea4b211a24b564117/roles/f5c487bc605f45df82ef6367c80fae70` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:04:59.898799 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea17274f-2206-4d81-b4fe-c6ab2c0382e4 None admin] RBAC: Authorizing `identity:create_grant(project_id=33bf210d015847439655b9943d1b0261, user_id=952911c492444ffea4b211a24b564117, role_id=f5c487bc605f45df82ef6367c80fae70)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:04:59.900398 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea17274f-2206-4d81-b4fe-c6ab2c0382e4 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:04:59.914826 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 310/612] 10.222.0.26 () {58 vars in 1439 bytes} [Thu Jul 14 09:04:59 2022] PUT /identity/v3/projects/33bf210d015847439655b9943d1b0261/users/952911c492444ffea4b211a24b564117/roles/f5c487bc605f45df82ef6367c80fae70 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:04:59.947560 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:04:59.983503 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bca35bc-0f69-46b8-a708-99c3da440255 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:00.030435 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16eb963f-f9f5-4b12-ad7f-357b97eaaede None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:00.031116 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16eb963f-f9f5-4b12-ad7f-357b97eaaede None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:00.031425 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16eb963f-f9f5-4b12-ad7f-357b97eaaede None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:00.031705 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16eb963f-f9f5-4b12-ad7f-357b97eaaede None admin] PATH_INFO: `/v3/domains` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:00.032444 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16eb963f-f9f5-4b12-ad7f-357b97eaaede None admin] RBAC: Adding query filter params (name=Default) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:05:00.032767 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16eb963f-f9f5-4b12-ad7f-357b97eaaede None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:00.034169 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16eb963f-f9f5-4b12-ad7f-357b97eaaede None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:00.048507 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 303/613] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Jul 14 09:04:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 09:05:00.053900 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea17274f-2206-4d81-b4fe-c6ab2c0382e4 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:00.169358 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:00.188182 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea17274f-2206-4d81-b4fe-c6ab2c0382e4 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:00.239467 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013361d8-4b0d-4955-8871-ece3bfabcce6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:00.240434 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013361d8-4b0d-4955-8871-ece3bfabcce6 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:00.240817 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013361d8-4b0d-4955-8871-ece3bfabcce6 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:00.241191 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013361d8-4b0d-4955-8871-ece3bfabcce6 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:00.253132 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013361d8-4b0d-4955-8871-ece3bfabcce6 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:00.254716 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013361d8-4b0d-4955-8871-ece3bfabcce6 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:00.276973 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-013361d8-4b0d-4955-8871-ece3bfabcce6 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 14 09:05:00.276973 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:05:00.276973 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:05:00.276973 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:05:00.276973 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:05:00.276973 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:05:00.276973 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:05:00.276973 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:05:00.276973 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:05:00.276973 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:05:00.276973 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:05:00.276973 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:05:00.276973 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 09:05:00.276973 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 09:05:00.276973 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 09:05:00.276973 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 09:05:00.278196 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:05:00.278196 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:05:00.278196 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 09:05:00.278196 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 09:05:00.278196 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 09:05:00.278196 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 09:05:00.278196 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:05:00.278196 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:05:00.278196 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:05:00.278196 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:05:00.278196 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:05:00.278196 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 14 09:05:00.278196 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:05:00.278196 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:05:00.278196 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:05:00.278196 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:05:00.278196 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:05:00.278196 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:05:00.279390 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:05:00.279390 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:05:00.279390 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 09:05:00.279390 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 09:05:00.279390 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 09:05:00.279390 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 09:05:00.279390 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 09:05:00.279390 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 09:05:00.279390 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 14 09:05:00.279390 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 14 09:05:00.280948 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 311/614] 10.222.0.26 () {56 vars in 1096 bytes} [Thu Jul 14 09:05:00 2022] GET /identity/v3/users/swift => generated 83 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:05:00.286673 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16eb963f-f9f5-4b12-ad7f-357b97eaaede None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:00.395000 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16eb963f-f9f5-4b12-ad7f-357b97eaaede None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:00.403453 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3698ff2-fcee-4553-ba92-f0c876d72ade None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:00.403863 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3698ff2-fcee-4553-ba92-f0c876d72ade None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:00.404240 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3698ff2-fcee-4553-ba92-f0c876d72ade None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:00.406723 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 312/615] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:05:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:05:00.412217 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ef1bec-7fe0-4bd2-8ae7-1ae0f9f4d455 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:00.412615 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ef1bec-7fe0-4bd2-8ae7-1ae0f9f4d455 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:00.412992 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ef1bec-7fe0-4bd2-8ae7-1ae0f9f4d455 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:00.432233 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-67ef1bec-7fe0-4bd2-8ae7-1ae0f9f4d455 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:05:00.433595 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-67ef1bec-7fe0-4bd2-8ae7-1ae0f9f4d455 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:05:00.465735 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295e53cb-f806-47bd-94e0-17f0f616b00c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:00.466416 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295e53cb-f806-47bd-94e0-17f0f616b00c None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:00.466701 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295e53cb-f806-47bd-94e0-17f0f616b00c None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:00.466980 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295e53cb-f806-47bd-94e0-17f0f616b00c None admin] PATH_INFO: `/v3/users` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:00.467860 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295e53cb-f806-47bd-94e0-17f0f616b00c None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:05:00.468170 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295e53cb-f806-47bd-94e0-17f0f616b00c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:00.469621 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295e53cb-f806-47bd-94e0-17f0f616b00c None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:00.488942 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:00.504788 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 304/616] 10.222.0.26 () {56 vars in 1135 bytes} [Thu Jul 14 09:05:00 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:05:00.511219 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295e53cb-f806-47bd-94e0-17f0f616b00c None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:00.531097 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 313/617] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:05:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 09:05:00.581226 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ef1bec-7fe0-4bd2-8ae7-1ae0f9f4d455 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:00.654498 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295e53cb-f806-47bd-94e0-17f0f616b00c None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:00.708880 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:00.720442 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e28e41-b3fc-4fb1-9951-1ddfcbe62125 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:00.721525 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e28e41-b3fc-4fb1-9951-1ddfcbe62125 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:00.722021 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e28e41-b3fc-4fb1-9951-1ddfcbe62125 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:00.722500 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e28e41-b3fc-4fb1-9951-1ddfcbe62125 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:00.732903 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ef1bec-7fe0-4bd2-8ae7-1ae0f9f4d455 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:00.745326 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e28e41-b3fc-4fb1-9951-1ddfcbe62125 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:00.746940 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e28e41-b3fc-4fb1-9951-1ddfcbe62125 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:00.782887 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-f0e28e41-b3fc-4fb1-9951-1ddfcbe62125 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 09:05:00.782887 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:05:00.782887 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:05:00.782887 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:05:00.782887 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:05:00.782887 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:05:00.782887 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:05:00.782887 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:05:00.782887 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:05:00.782887 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:05:00.782887 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:05:00.782887 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:05:00.782887 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 09:05:00.782887 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 09:05:00.782887 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 09:05:00.784033 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 09:05:00.784033 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:05:00.784033 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:05:00.784033 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:05:00.784033 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:05:00.784033 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:05:00.784033 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:05:00.784033 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:05:00.784033 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 14 09:05:00.784033 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:05:00.784033 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:05:00.784033 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:05:00.784033 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:05:00.784033 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:05:00.784033 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:05:00.784033 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:05:00.784033 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:05:00.785143 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 09:05:00.785143 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 09:05:00.785143 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 09:05:00.785143 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 14 09:05:00.785143 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 305/618] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jul 14 09:05:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 09:05:00.790218 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae7ccee-cbd8-4014-ab15-0cf7fbadb97e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:00.791377 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae7ccee-cbd8-4014-ab15-0cf7fbadb97e None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:00.791687 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae7ccee-cbd8-4014-ab15-0cf7fbadb97e None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:00.791959 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae7ccee-cbd8-4014-ab15-0cf7fbadb97e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:00.799146 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e28e41-b3fc-4fb1-9951-1ddfcbe62125 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:00.806708 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae7ccee-cbd8-4014-ab15-0cf7fbadb97e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 09:05:00.814512 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-4ae7ccee-cbd8-4014-ab15-0cf7fbadb97e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 09:05:00.814512 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:05:00.814512 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:05:00.814512 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:05:00.814512 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:05:00.814512 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:05:00.814512 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:05:00.814512 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:05:00.814512 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:05:00.814512 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:05:00.814512 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:05:00.814512 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:05:00.814512 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 09:05:00.814512 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 09:05:00.814512 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 09:05:00.814512 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 09:05:00.816011 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 09:05:00.816011 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 09:05:00.816011 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 09:05:00.816011 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 09:05:00.816011 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:05:00.816011 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:05:00.816011 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:05:00.816011 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:05:00.816011 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:05:00.816011 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:05:00.816011 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:05:00.816011 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 14 09:05:00.816011 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:05:00.816011 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:05:00.816011 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:05:00.816011 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:05:00.819114 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:05:00.819114 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:05:00.819114 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:05:00.819114 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:05:00.819114 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 09:05:00.819114 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 09:05:00.819114 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 09:05:00.819114 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 09:05:00.819114 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 09:05:00.819114 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 09:05:00.819114 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 09:05:00.819114 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 14 09:05:00.819114 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 314/619] 10.222.0.26 () {56 vars in 1102 bytes} [Thu Jul 14 09:05:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:05:00.825817 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae7ccee-cbd8-4014-ab15-0cf7fbadb97e None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:00.891694 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:00.982958 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e28e41-b3fc-4fb1-9951-1ddfcbe62125 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:01.013435 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae7ccee-cbd8-4014-ab15-0cf7fbadb97e None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:01.044415 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10abbca8-ea8b-4c33-a8b0-5ec6fa07ea63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:01.045795 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10abbca8-ea8b-4c33-a8b0-5ec6fa07ea63 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:01.046328 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10abbca8-ea8b-4c33-a8b0-5ec6fa07ea63 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:01.046809 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10abbca8-ea8b-4c33-a8b0-5ec6fa07ea63 None admin] PATH_INFO: `/v3/domains` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:01.047766 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10abbca8-ea8b-4c33-a8b0-5ec6fa07ea63 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:05:01.048282 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10abbca8-ea8b-4c33-a8b0-5ec6fa07ea63 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:01.049890 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10abbca8-ea8b-4c33-a8b0-5ec6fa07ea63 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:01.062739 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 306/620] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Jul 14 09:05:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:05:01.068400 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10abbca8-ea8b-4c33-a8b0-5ec6fa07ea63 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:01.079439 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b16df1-69b8-4777-bdcb-c3e78914a490 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:01.080245 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b16df1-69b8-4777-bdcb-c3e78914a490 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:01.080495 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b16df1-69b8-4777-bdcb-c3e78914a490 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:01.080777 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b16df1-69b8-4777-bdcb-c3e78914a490 None admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:01.081540 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b16df1-69b8-4777-bdcb-c3e78914a490 None admin] RBAC: Adding query filter params (name=service) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:05:01.081906 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b16df1-69b8-4777-bdcb-c3e78914a490 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:01.083409 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b16df1-69b8-4777-bdcb-c3e78914a490 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:01.095898 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:01.103657 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 315/621] 10.222.0.26 () {56 vars in 1103 bytes} [Thu Jul 14 09:05:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 09:05:01.107688 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b16df1-69b8-4777-bdcb-c3e78914a490 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:01.184850 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10abbca8-ea8b-4c33-a8b0-5ec6fa07ea63 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:01.195864 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:01.223476 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbfb91b-33bd-4d49-bac6-69f5ac7f820c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:01.224435 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbfb91b-33bd-4d49-bac6-69f5ac7f820c None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:01.224825 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbfb91b-33bd-4d49-bac6-69f5ac7f820c None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:01.225225 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbfb91b-33bd-4d49-bac6-69f5ac7f820c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:01.233451 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbfb91b-33bd-4d49-bac6-69f5ac7f820c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:01.235318 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbfb91b-33bd-4d49-bac6-69f5ac7f820c None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:01.251036 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-acbfb91b-33bd-4d49-bac6-69f5ac7f820c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 09:05:01.251036 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:05:01.251036 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:05:01.251036 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:05:01.251036 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:05:01.251036 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:05:01.251036 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:05:01.251036 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:05:01.251036 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:05:01.251036 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:05:01.251036 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:05:01.251036 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:05:01.251036 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 09:05:01.251036 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 09:05:01.251036 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 09:05:01.252169 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 09:05:01.252169 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:05:01.252169 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:05:01.252169 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:05:01.252169 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:05:01.252169 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:05:01.252169 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:05:01.252169 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:05:01.252169 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 14 09:05:01.252169 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:05:01.252169 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:05:01.252169 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:05:01.252169 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:05:01.252169 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:05:01.252169 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:05:01.252169 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:05:01.252169 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:05:01.253298 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 09:05:01.253298 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 09:05:01.253298 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 09:05:01.253298 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 09:05:01.253298 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 09:05:01.253298 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 09:05:01.253298 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 09:05:01.253298 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 14 09:05:01.255315 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 307/622] 10.222.0.26 () {56 vars in 1111 bytes} [Thu Jul 14 09:05:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 09:05:01.261069 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbfb91b-33bd-4d49-bac6-69f5ac7f820c None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:01.367181 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b16df1-69b8-4777-bdcb-c3e78914a490 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:01.398953 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbfb91b-33bd-4d49-bac6-69f5ac7f820c None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:01.414434 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ded34b-ffd7-4eb9-a886-f4bcf934e02e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:01.415589 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ded34b-ffd7-4eb9-a886-f4bcf934e02e None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:01.415875 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ded34b-ffd7-4eb9-a886-f4bcf934e02e None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:01.416150 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ded34b-ffd7-4eb9-a886-f4bcf934e02e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:01.440983 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ded34b-ffd7-4eb9-a886-f4bcf934e02e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:01.443862 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ded34b-ffd7-4eb9-a886-f4bcf934e02e None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:01.466388 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b817d52e-41ae-408e-853c-763a8d495148 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:01.467064 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b817d52e-41ae-408e-853c-763a8d495148 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:01.467384 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b817d52e-41ae-408e-853c-763a8d495148 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:01.467663 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b817d52e-41ae-408e-853c-763a8d495148 None admin] PATH_INFO: `/v3/projects` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:01.468462 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b817d52e-41ae-408e-853c-763a8d495148 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:05:01.468771 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b817d52e-41ae-408e-853c-763a8d495148 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:01.470230 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b817d52e-41ae-408e-853c-763a8d495148 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:01.480873 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-09ded34b-ffd7-4eb9-a886-f4bcf934e02e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 09:05:01.480873 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:05:01.480873 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:05:01.480873 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:05:01.480873 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:05:01.480873 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:05:01.480873 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:05:01.480873 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:05:01.480873 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:05:01.480873 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:05:01.480873 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:05:01.480873 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:05:01.480873 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 09:05:01.480873 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 09:05:01.480873 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 09:05:01.481941 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 09:05:01.481941 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:05:01.481941 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:05:01.481941 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:05:01.481941 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:05:01.481941 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:05:01.481941 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:05:01.481941 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:05:01.481941 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 14 09:05:01.481941 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:05:01.481941 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:05:01.481941 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:05:01.481941 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:05:01.481941 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:05:01.481941 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:05:01.481941 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:05:01.481941 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:05:01.483200 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 09:05:01.483200 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 09:05:01.483200 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 09:05:01.483200 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 14 09:05:01.483200 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 316/623] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jul 14 09:05:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 375 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 09:05:01.486181 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6fa468-5760-4450-9420-5a4ce237849c None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:01.486181 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6fa468-5760-4450-9420-5a4ce237849c None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:01.486181 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6fa468-5760-4450-9420-5a4ce237849c None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:01.487132 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 317/624] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:05:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:05:01.487858 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6fa468-5760-4450-9420-5a4ce237849c None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:01.493563 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 308/625] 10.222.0.26 () {56 vars in 1148 bytes} [Thu Jul 14 09:05:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:05:01.495098 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63485618-3dc8-42fd-813f-04c0ec40070f None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:01.495590 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63485618-3dc8-42fd-813f-04c0ec40070f None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:01.495986 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63485618-3dc8-42fd-813f-04c0ec40070f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:01.512755 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-63485618-3dc8-42fd-813f-04c0ec40070f None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:05:01.514122 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-63485618-3dc8-42fd-813f-04c0ec40070f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:05:01.539257 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:01.559118 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:01.597663 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 309/626] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:05:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 09:05:01.598760 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63485618-3dc8-42fd-813f-04c0ec40070f None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:01.635616 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6fa468-5760-4450-9420-5a4ce237849c None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:01.671559 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311c630b-34e5-4eb9-88e8-a65aeb099435 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:01.672388 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311c630b-34e5-4eb9-88e8-a65aeb099435 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:01.672778 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311c630b-34e5-4eb9-88e8-a65aeb099435 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:01.673175 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311c630b-34e5-4eb9-88e8-a65aeb099435 None admin] PATH_INFO: `/v3/domains` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:01.674030 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311c630b-34e5-4eb9-88e8-a65aeb099435 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:05:01.674443 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311c630b-34e5-4eb9-88e8-a65aeb099435 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:01.675954 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311c630b-34e5-4eb9-88e8-a65aeb099435 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:01.688258 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 318/627] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Jul 14 09:05:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:05:01.689207 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311c630b-34e5-4eb9-88e8-a65aeb099435 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:01.726512 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63485618-3dc8-42fd-813f-04c0ec40070f None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:01.766703 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34950aa-5160-4e7a-8802-739e26e2c164 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:01.767681 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34950aa-5160-4e7a-8802-739e26e2c164 None admin] REQUEST_METHOD: `PUT` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:01.767967 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34950aa-5160-4e7a-8802-739e26e2c164 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:01.768365 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34950aa-5160-4e7a-8802-739e26e2c164 None admin] PATH_INFO: `/v3/projects/33bf210d015847439655b9943d1b0261/users/e5a43c80514b4a13af03c7f8ebcf7a5f/roles/f5c487bc605f45df82ef6367c80fae70` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:01.785532 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34950aa-5160-4e7a-8802-739e26e2c164 None admin] RBAC: Authorizing `identity:create_grant(project_id=33bf210d015847439655b9943d1b0261, user_id=e5a43c80514b4a13af03c7f8ebcf7a5f, role_id=f5c487bc605f45df82ef6367c80fae70)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:01.786974 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34950aa-5160-4e7a-8802-739e26e2c164 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:01.802167 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 310/628] 10.222.0.26 () {58 vars in 1439 bytes} [Thu Jul 14 09:05:01 2022] PUT /identity/v3/projects/33bf210d015847439655b9943d1b0261/users/e5a43c80514b4a13af03c7f8ebcf7a5f/roles/f5c487bc605f45df82ef6367c80fae70 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:05:01.803027 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34950aa-5160-4e7a-8802-739e26e2c164 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:01.814332 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311c630b-34e5-4eb9-88e8-a65aeb099435 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:01.820623 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:01.840327 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:01.867059 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33efdc3a-a4cf-416c-b6f6-7551318a22aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:01.868381 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33efdc3a-a4cf-416c-b6f6-7551318a22aa None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:01.868880 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33efdc3a-a4cf-416c-b6f6-7551318a22aa None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:01.870110 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33efdc3a-a4cf-416c-b6f6-7551318a22aa None admin] PATH_INFO: `/v3/roles/service` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:01.888086 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33efdc3a-a4cf-416c-b6f6-7551318a22aa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 09:05:01.899431 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-33efdc3a-a4cf-416c-b6f6-7551318a22aa None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 09:05:01.899431 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:05:01.899431 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:05:01.899431 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:05:01.899431 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:05:01.899431 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:05:01.899431 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:05:01.899431 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:05:01.899431 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:05:01.899431 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:05:01.899431 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:05:01.899431 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:05:01.899431 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 09:05:01.899431 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 09:05:01.899431 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 09:05:01.899431 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 09:05:01.900550 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 09:05:01.900550 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 09:05:01.900550 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 09:05:01.900550 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 09:05:01.900550 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:05:01.900550 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:05:01.900550 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:05:01.900550 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:05:01.900550 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:05:01.900550 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:05:01.900550 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:05:01.900550 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 14 09:05:01.900550 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:05:01.900550 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:05:01.900550 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:05:01.900550 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:05:01.901445 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:05:01.901445 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:05:01.901445 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:05:01.901445 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:05:01.901445 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 09:05:01.901445 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 09:05:01.901445 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 09:05:01.901445 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 09:05:01.901445 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 09:05:01.901445 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 09:05:01.901445 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 09:05:01.901445 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 14 09:05:01.904618 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 319/629] 10.222.0.26 () {56 vars in 1102 bytes} [Thu Jul 14 09:05:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:05:01.906530 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0108e9cd-8736-436b-af6e-c119529657a2 None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:01.906969 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0108e9cd-8736-436b-af6e-c119529657a2 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:01.907996 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0108e9cd-8736-436b-af6e-c119529657a2 None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:01.910084 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 320/630] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:05:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:05:01.917802 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa69341d-6c9b-4e56-98bb-42ec21ab240a None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:01.918336 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa69341d-6c9b-4e56-98bb-42ec21ab240a None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:01.918758 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa69341d-6c9b-4e56-98bb-42ec21ab240a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:01.934178 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34950aa-5160-4e7a-8802-739e26e2c164 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:01.939330 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-fa69341d-6c9b-4e56-98bb-42ec21ab240a None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:05:01.940808 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-fa69341d-6c9b-4e56-98bb-42ec21ab240a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:05:01.980780 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcb5756-0cd2-4e61-b548-f616d12ebb59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:01.981747 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcb5756-0cd2-4e61-b548-f616d12ebb59 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:01.982151 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcb5756-0cd2-4e61-b548-f616d12ebb59 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:01.982524 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcb5756-0cd2-4e61-b548-f616d12ebb59 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:01.992834 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcb5756-0cd2-4e61-b548-f616d12ebb59 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:01.994862 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcb5756-0cd2-4e61-b548-f616d12ebb59 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:02.015069 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-8fcb5756-0cd2-4e61-b548-f616d12ebb59 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 14 09:05:02.015069 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:05:02.015069 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:05:02.015069 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:05:02.015069 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:05:02.015069 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:05:02.015069 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:05:02.015069 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:05:02.015069 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:05:02.015069 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:05:02.015069 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:05:02.015069 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:05:02.015069 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 09:05:02.015069 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 09:05:02.015069 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 09:05:02.015069 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 09:05:02.016107 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:05:02.016107 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:05:02.016107 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 09:05:02.016107 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 09:05:02.016107 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 09:05:02.016107 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 09:05:02.016107 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:05:02.016107 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:05:02.016107 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:05:02.016107 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:05:02.016107 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:05:02.016107 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 14 09:05:02.016107 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:05:02.016107 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:05:02.016107 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:05:02.016107 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:05:02.016107 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:05:02.016107 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:05:02.017105 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:05:02.017105 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:05:02.017105 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 09:05:02.017105 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 09:05:02.017105 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 09:05:02.017105 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 09:05:02.017105 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 09:05:02.017105 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 09:05:02.017105 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 14 09:05:02.017105 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 14 09:05:02.020107 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 311/631] 10.222.0.26 () {56 vars in 1093 bytes} [Thu Jul 14 09:05:01 2022] GET /identity/v3/users/nova => generated 82 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:05:02.021293 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcb5756-0cd2-4e61-b548-f616d12ebb59 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:02.045470 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 321/632] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:05:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 09:05:02.046460 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa69341d-6c9b-4e56-98bb-42ec21ab240a None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:02.100544 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:02.142677 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcb5756-0cd2-4e61-b548-f616d12ebb59 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:02.180574 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa69341d-6c9b-4e56-98bb-42ec21ab240a None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:02.182508 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:02.184941 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ad2bea-f6b2-4370-acbb-498a6b7b2685 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:02.185814 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ad2bea-f6b2-4370-acbb-498a6b7b2685 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:02.186236 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ad2bea-f6b2-4370-acbb-498a6b7b2685 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:02.186741 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ad2bea-f6b2-4370-acbb-498a6b7b2685 None admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:02.187722 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ad2bea-f6b2-4370-acbb-498a6b7b2685 None admin] RBAC: Adding query filter params (name=service) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:05:02.188216 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ad2bea-f6b2-4370-acbb-498a6b7b2685 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:02.189732 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ad2bea-f6b2-4370-acbb-498a6b7b2685 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:02.202056 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 312/633] 10.222.0.26 () {56 vars in 1103 bytes} [Thu Jul 14 09:05:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:05:02.203097 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ad2bea-f6b2-4370-acbb-498a6b7b2685 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:02.219689 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c9291a-bfee-4d6c-8091-d983356eb98e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:02.220529 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c9291a-bfee-4d6c-8091-d983356eb98e None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:02.220898 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c9291a-bfee-4d6c-8091-d983356eb98e None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:02.221273 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c9291a-bfee-4d6c-8091-d983356eb98e None admin] PATH_INFO: `/v3/users` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:02.222266 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c9291a-bfee-4d6c-8091-d983356eb98e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:05:02.222671 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c9291a-bfee-4d6c-8091-d983356eb98e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:02.224278 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c9291a-bfee-4d6c-8091-d983356eb98e None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:02.245811 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 322/634] 10.222.0.26 () {56 vars in 1133 bytes} [Thu Jul 14 09:05:02 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:05:02.246693 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c9291a-bfee-4d6c-8091-d983356eb98e None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:02.314106 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ad2bea-f6b2-4370-acbb-498a6b7b2685 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:02.344915 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:02.349527 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c9291a-bfee-4d6c-8091-d983356eb98e None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:02.352578 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48fe09c-f8ba-4129-b491-981a3795a9bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:02.353484 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48fe09c-f8ba-4129-b491-981a3795a9bd None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:02.353907 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48fe09c-f8ba-4129-b491-981a3795a9bd None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:02.354276 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48fe09c-f8ba-4129-b491-981a3795a9bd None admin] PATH_INFO: `/v3/roles/service` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:02.365285 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48fe09c-f8ba-4129-b491-981a3795a9bd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 09:05:02.373874 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-d48fe09c-f8ba-4129-b491-981a3795a9bd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 09:05:02.373874 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:05:02.373874 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:05:02.373874 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:05:02.373874 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:05:02.373874 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:05:02.373874 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:05:02.373874 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:05:02.373874 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:05:02.373874 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:05:02.373874 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:05:02.373874 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:05:02.373874 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 09:05:02.373874 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 09:05:02.373874 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 09:05:02.373874 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 09:05:02.375033 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 09:05:02.375033 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 09:05:02.375033 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 09:05:02.375033 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 09:05:02.375033 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:05:02.375033 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:05:02.375033 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:05:02.375033 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:05:02.375033 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:05:02.375033 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:05:02.375033 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:05:02.375033 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 14 09:05:02.375033 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:05:02.375033 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:05:02.375033 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:05:02.375033 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:05:02.376097 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:05:02.376097 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:05:02.376097 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:05:02.376097 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:05:02.376097 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 09:05:02.376097 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 09:05:02.376097 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 09:05:02.376097 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 09:05:02.376097 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 09:05:02.376097 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 09:05:02.376097 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 09:05:02.376097 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 14 09:05:02.377886 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 313/635] 10.222.0.26 () {56 vars in 1102 bytes} [Thu Jul 14 09:05:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:05:02.379226 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d166c3-5c3e-401b-b8b0-bd577f83aa33 None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:02.379606 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d166c3-5c3e-401b-b8b0-bd577f83aa33 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:02.379976 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d166c3-5c3e-401b-b8b0-bd577f83aa33 None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:02.385400 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 314/636] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:05:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:05:02.385400 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d166c3-5c3e-401b-b8b0-bd577f83aa33 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:02.394949 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b82873e-1051-4c9a-a74e-8d561ccef5fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:02.396298 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b82873e-1051-4c9a-a74e-8d561ccef5fd None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:02.396579 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b82873e-1051-4c9a-a74e-8d561ccef5fd None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:02.396855 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b82873e-1051-4c9a-a74e-8d561ccef5fd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:02.407944 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b82873e-1051-4c9a-a74e-8d561ccef5fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:02.409494 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b82873e-1051-4c9a-a74e-8d561ccef5fd None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:02.427709 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-3b82873e-1051-4c9a-a74e-8d561ccef5fd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 09:05:02.427709 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:05:02.427709 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:05:02.427709 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:05:02.427709 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:05:02.427709 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:05:02.427709 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:05:02.427709 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:05:02.427709 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:05:02.427709 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:05:02.427709 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:05:02.427709 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:05:02.427709 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 09:05:02.427709 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 09:05:02.427709 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 09:05:02.428638 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 09:05:02.428638 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:05:02.428638 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:05:02.428638 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:05:02.428638 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:05:02.428638 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:05:02.428638 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:05:02.428638 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:05:02.428638 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 14 09:05:02.428638 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:05:02.428638 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:05:02.428638 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:05:02.428638 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:05:02.428638 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:05:02.428638 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:05:02.428638 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:05:02.428638 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:05:02.429519 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 09:05:02.429519 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 09:05:02.429519 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 09:05:02.429519 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 14 09:05:02.431506 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 323/637] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jul 14 09:05:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:05:02.432874 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b82873e-1051-4c9a-a74e-8d561ccef5fd None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:02.448014 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:02.494157 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d166c3-5c3e-401b-b8b0-bd577f83aa33 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:02.531735 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4112524a-e94f-4f2e-a31f-d1ae2b70879c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:02.532386 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4112524a-e94f-4f2e-a31f-d1ae2b70879c None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:02.532670 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4112524a-e94f-4f2e-a31f-d1ae2b70879c None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:02.532947 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4112524a-e94f-4f2e-a31f-d1ae2b70879c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:02.539340 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b82873e-1051-4c9a-a74e-8d561ccef5fd None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:02.573316 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f00d41-41ed-4665-903c-f95606344af9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:02.573991 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f00d41-41ed-4665-903c-f95606344af9 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:02.574270 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f00d41-41ed-4665-903c-f95606344af9 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:02.574538 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f00d41-41ed-4665-903c-f95606344af9 None admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:02.575315 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f00d41-41ed-4665-903c-f95606344af9 None admin] RBAC: Adding query filter params (name=service) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:05:02.575619 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f00d41-41ed-4665-903c-f95606344af9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:02.577000 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f00d41-41ed-4665-903c-f95606344af9 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:02.585726 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 324/638] 10.222.0.26 () {56 vars in 1103 bytes} [Thu Jul 14 09:05:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:05:02.586922 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814d1244-4c2f-4bfa-ad8e-0eb4b5c84d8d None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:02.587226 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814d1244-4c2f-4bfa-ad8e-0eb4b5c84d8d None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:02.587522 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814d1244-4c2f-4bfa-ad8e-0eb4b5c84d8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:02.603548 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-814d1244-4c2f-4bfa-ad8e-0eb4b5c84d8d None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:05:02.603761 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:02.604782 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-814d1244-4c2f-4bfa-ad8e-0eb4b5c84d8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:05:02.615447 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4112524a-e94f-4f2e-a31f-d1ae2b70879c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:02.617104 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4112524a-e94f-4f2e-a31f-d1ae2b70879c None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:02.672624 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 325/639] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:05:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 09:05:02.673377 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814d1244-4c2f-4bfa-ad8e-0eb4b5c84d8d None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:02.679589 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:02.734406 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 315/640] 10.222.0.26 () {58 vars in 1282 bytes} [Thu Jul 14 09:05:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 09:05:02.735694 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4112524a-e94f-4f2e-a31f-d1ae2b70879c None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:02.776181 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814d1244-4c2f-4bfa-ad8e-0eb4b5c84d8d None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:02.809872 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c5cf62-16f0-44ee-b088-cb15c96d0322 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:02.810531 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c5cf62-16f0-44ee-b088-cb15c96d0322 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:02.810810 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c5cf62-16f0-44ee-b088-cb15c96d0322 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:02.811078 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c5cf62-16f0-44ee-b088-cb15c96d0322 None admin] PATH_INFO: `/v3/domains` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:02.813127 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c5cf62-16f0-44ee-b088-cb15c96d0322 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:05:02.813428 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c5cf62-16f0-44ee-b088-cb15c96d0322 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:02.814790 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c5cf62-16f0-44ee-b088-cb15c96d0322 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:02.826442 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 326/641] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Jul 14 09:05:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:05:02.827249 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c5cf62-16f0-44ee-b088-cb15c96d0322 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:02.834514 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:02.848126 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4112524a-e94f-4f2e-a31f-d1ae2b70879c None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:02.885314 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bedd4b3-75d7-4a5b-82ed-1cc515cb3ba6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:02.886567 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bedd4b3-75d7-4a5b-82ed-1cc515cb3ba6 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:02.886875 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bedd4b3-75d7-4a5b-82ed-1cc515cb3ba6 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:02.887146 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bedd4b3-75d7-4a5b-82ed-1cc515cb3ba6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:02.889530 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:02.899346 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bedd4b3-75d7-4a5b-82ed-1cc515cb3ba6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:02.900891 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bedd4b3-75d7-4a5b-82ed-1cc515cb3ba6 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:02.918892 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-3bedd4b3-75d7-4a5b-82ed-1cc515cb3ba6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 09:05:02.918892 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:05:02.918892 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:05:02.918892 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:05:02.918892 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:05:02.918892 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:05:02.918892 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:05:02.918892 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:05:02.918892 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:05:02.918892 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:05:02.918892 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:05:02.918892 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:05:02.918892 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 09:05:02.918892 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 09:05:02.918892 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 09:05:02.920008 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 09:05:02.920008 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:05:02.920008 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:05:02.920008 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:05:02.920008 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:05:02.920008 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:05:02.920008 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:05:02.920008 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:05:02.920008 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 14 09:05:02.920008 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:05:02.920008 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:05:02.920008 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:05:02.920008 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:05:02.920008 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:05:02.920008 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:05:02.920008 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:05:02.920008 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:05:02.921113 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 09:05:02.921113 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 09:05:02.921113 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 09:05:02.921113 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 14 09:05:02.923390 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 316/642] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jul 14 09:05:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:05:02.924941 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bedd4b3-75d7-4a5b-82ed-1cc515cb3ba6 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:02.933021 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c5cf62-16f0-44ee-b088-cb15c96d0322 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:02.971151 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae6ce73-585d-4e55-90b7-d4eadc4ebf90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:02.972299 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae6ce73-585d-4e55-90b7-d4eadc4ebf90 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:02.972581 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae6ce73-585d-4e55-90b7-d4eadc4ebf90 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:02.972855 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae6ce73-585d-4e55-90b7-d4eadc4ebf90 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:02.983948 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae6ce73-585d-4e55-90b7-d4eadc4ebf90 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 09:05:02.991558 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-5ae6ce73-585d-4e55-90b7-d4eadc4ebf90 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 09:05:02.991558 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:05:02.991558 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:05:02.991558 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:05:02.991558 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:05:02.991558 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:05:02.991558 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:05:02.991558 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:05:02.991558 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:05:02.991558 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:05:02.991558 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:05:02.991558 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:05:02.991558 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 09:05:02.991558 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 09:05:02.991558 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 09:05:02.991558 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 09:05:02.992564 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 09:05:02.992564 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 09:05:02.992564 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 09:05:02.992564 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 09:05:02.992564 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:05:02.992564 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:05:02.992564 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:05:02.992564 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:05:02.992564 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:05:02.992564 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:05:02.992564 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:05:02.992564 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 14 09:05:02.992564 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:05:02.992564 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:05:02.992564 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:05:02.992564 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:05:02.993399 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:05:02.993399 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:05:02.993399 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:05:02.993399 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:05:02.993399 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 09:05:02.993399 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 09:05:02.993399 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 09:05:02.993399 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 09:05:02.993399 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 09:05:02.993399 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 09:05:02.993399 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 09:05:02.993399 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 14 09:05:02.993399 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 327/643] 10.222.0.26 () {56 vars in 1102 bytes} [Thu Jul 14 09:05:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:05:02.994085 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae6ce73-585d-4e55-90b7-d4eadc4ebf90 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:03.039608 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bedd4b3-75d7-4a5b-82ed-1cc515cb3ba6 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:03.073497 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b072f0-e98e-4ef2-98ab-8943e3deb757 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:03.074393 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b072f0-e98e-4ef2-98ab-8943e3deb757 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:03.074703 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b072f0-e98e-4ef2-98ab-8943e3deb757 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:03.075097 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b072f0-e98e-4ef2-98ab-8943e3deb757 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:03.085615 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b072f0-e98e-4ef2-98ab-8943e3deb757 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:03.087114 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b072f0-e98e-4ef2-98ab-8943e3deb757 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:03.089108 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:03.104874 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-51b072f0-e98e-4ef2-98ab-8943e3deb757 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 09:05:03.104874 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:05:03.104874 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:05:03.104874 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:05:03.104874 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:05:03.104874 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:05:03.104874 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:05:03.104874 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:05:03.104874 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:05:03.104874 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:05:03.104874 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:05:03.104874 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:05:03.104874 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 09:05:03.104874 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 09:05:03.104874 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 09:05:03.113116 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 09:05:03.113116 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:05:03.113116 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:05:03.113116 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:05:03.113116 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:05:03.113116 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:05:03.113116 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:05:03.113116 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:05:03.113116 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 14 09:05:03.113116 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:05:03.113116 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:05:03.113116 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:05:03.113116 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:05:03.113116 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:05:03.113116 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:05:03.113116 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:05:03.113116 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:05:03.114853 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 09:05:03.114853 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 09:05:03.114853 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 09:05:03.114853 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 14 09:05:03.114853 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 317/644] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jul 14 09:05:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:05:03.116109 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae6ce73-585d-4e55-90b7-d4eadc4ebf90 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:03.116219 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b072f0-e98e-4ef2-98ab-8943e3deb757 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:03.137504 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:03.149500 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7bbb1e-299c-4005-9007-a72f0b767fcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:03.150570 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7bbb1e-299c-4005-9007-a72f0b767fcd None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:03.150913 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7bbb1e-299c-4005-9007-a72f0b767fcd None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:03.151390 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7bbb1e-299c-4005-9007-a72f0b767fcd None admin] PATH_INFO: `/v3/users/glance` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:03.159631 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7bbb1e-299c-4005-9007-a72f0b767fcd None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:03.161256 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7bbb1e-299c-4005-9007-a72f0b767fcd None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:03.177189 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-ce7bbb1e-299c-4005-9007-a72f0b767fcd None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 14 09:05:03.177189 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:05:03.177189 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:05:03.177189 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:05:03.177189 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:05:03.177189 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:05:03.177189 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:05:03.177189 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:05:03.177189 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:05:03.177189 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:05:03.177189 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:05:03.177189 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:05:03.177189 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 09:05:03.177189 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 09:05:03.177189 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 09:05:03.177189 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 09:05:03.178295 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:05:03.178295 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:05:03.178295 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 09:05:03.178295 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 09:05:03.178295 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 09:05:03.178295 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 09:05:03.178295 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:05:03.178295 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:05:03.178295 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:05:03.178295 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:05:03.178295 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:05:03.178295 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 14 09:05:03.178295 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:05:03.178295 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:05:03.178295 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:05:03.178295 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:05:03.178295 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:05:03.178295 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:05:03.179403 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:05:03.179403 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:05:03.179403 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 09:05:03.179403 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 09:05:03.179403 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 09:05:03.179403 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 09:05:03.179403 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 09:05:03.179403 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 09:05:03.179403 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 14 09:05:03.179403 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 14 09:05:03.179403 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 328/645] 10.222.0.26 () {56 vars in 1099 bytes} [Thu Jul 14 09:05:02 2022] GET /identity/v3/users/glance => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:05:03.180000 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7bbb1e-299c-4005-9007-a72f0b767fcd None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:03.226183 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b072f0-e98e-4ef2-98ab-8943e3deb757 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:03.261533 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61da1b40-cd68-4d8b-be4d-524b2d76a7eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:03.262309 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61da1b40-cd68-4d8b-be4d-524b2d76a7eb None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:03.262930 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61da1b40-cd68-4d8b-be4d-524b2d76a7eb None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:03.262930 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61da1b40-cd68-4d8b-be4d-524b2d76a7eb None admin] PATH_INFO: `/v3/domains` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:03.263823 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61da1b40-cd68-4d8b-be4d-524b2d76a7eb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:05:03.264239 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61da1b40-cd68-4d8b-be4d-524b2d76a7eb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:03.265637 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61da1b40-cd68-4d8b-be4d-524b2d76a7eb None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:03.282813 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 318/646] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Jul 14 09:05:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:05:03.282813 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61da1b40-cd68-4d8b-be4d-524b2d76a7eb None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:03.285425 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7bbb1e-299c-4005-9007-a72f0b767fcd None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:03.321628 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b2c17b-5af2-4db2-b236-5c6d1c300f83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:03.322451 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b2c17b-5af2-4db2-b236-5c6d1c300f83 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:03.322795 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b2c17b-5af2-4db2-b236-5c6d1c300f83 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:03.323135 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b2c17b-5af2-4db2-b236-5c6d1c300f83 None admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:03.323977 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b2c17b-5af2-4db2-b236-5c6d1c300f83 None admin] RBAC: Adding query filter params (name=service) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:05:03.324348 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b2c17b-5af2-4db2-b236-5c6d1c300f83 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:03.325787 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b2c17b-5af2-4db2-b236-5c6d1c300f83 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:03.334651 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 329/647] 10.222.0.26 () {56 vars in 1103 bytes} [Thu Jul 14 09:05:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:05:03.335670 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b2c17b-5af2-4db2-b236-5c6d1c300f83 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:03.343344 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:03.384482 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:03.394282 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61da1b40-cd68-4d8b-be4d-524b2d76a7eb None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:03.429290 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f9e83e-801d-45be-a68e-3a96feeda47b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:03.430017 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f9e83e-801d-45be-a68e-3a96feeda47b None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:03.430427 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f9e83e-801d-45be-a68e-3a96feeda47b None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:03.430800 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f9e83e-801d-45be-a68e-3a96feeda47b None admin] PATH_INFO: `/v3/domains` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:03.431707 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f9e83e-801d-45be-a68e-3a96feeda47b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:05:03.432091 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f9e83e-801d-45be-a68e-3a96feeda47b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:03.433563 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f9e83e-801d-45be-a68e-3a96feeda47b None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:03.440269 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b2c17b-5af2-4db2-b236-5c6d1c300f83 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:03.446331 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 319/648] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Jul 14 09:05:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:05:03.447634 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f9e83e-801d-45be-a68e-3a96feeda47b None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:03.484550 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62fd225-63ea-42dd-9caa-86aae0e17119 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:03.485407 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62fd225-63ea-42dd-9caa-86aae0e17119 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:03.485779 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62fd225-63ea-42dd-9caa-86aae0e17119 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:03.486148 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62fd225-63ea-42dd-9caa-86aae0e17119 None admin] PATH_INFO: `/v3/users` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:03.487092 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62fd225-63ea-42dd-9caa-86aae0e17119 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:05:03.487521 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62fd225-63ea-42dd-9caa-86aae0e17119 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:03.489177 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62fd225-63ea-42dd-9caa-86aae0e17119 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:03.508528 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 330/649] 10.222.0.26 () {56 vars in 1137 bytes} [Thu Jul 14 09:05:03 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:05:03.509544 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62fd225-63ea-42dd-9caa-86aae0e17119 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:03.559787 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f9e83e-801d-45be-a68e-3a96feeda47b None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:03.574490 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:03.598353 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1d3770-0a9d-447e-af23-abc40e378cbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:03.599144 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1d3770-0a9d-447e-af23-abc40e378cbc None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:03.599453 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1d3770-0a9d-447e-af23-abc40e378cbc None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:03.599723 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1d3770-0a9d-447e-af23-abc40e378cbc None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:03.607464 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1d3770-0a9d-447e-af23-abc40e378cbc None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:03.608907 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1d3770-0a9d-447e-af23-abc40e378cbc None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:03.620920 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62fd225-63ea-42dd-9caa-86aae0e17119 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:03.623158 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-9b1d3770-0a9d-447e-af23-abc40e378cbc None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 14 09:05:03.623158 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:05:03.623158 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:05:03.623158 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:05:03.623158 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:05:03.623158 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:05:03.623158 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:05:03.623158 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:05:03.623158 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:05:03.623158 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:05:03.623158 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:05:03.623158 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:05:03.623158 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 09:05:03.623158 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 09:05:03.623158 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 09:05:03.623158 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 09:05:03.624293 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:05:03.624293 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:05:03.624293 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 09:05:03.624293 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 09:05:03.624293 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 09:05:03.624293 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 09:05:03.624293 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:05:03.624293 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:05:03.624293 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:05:03.624293 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:05:03.624293 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:05:03.624293 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 14 09:05:03.624293 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:05:03.624293 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:05:03.624293 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:05:03.624293 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:05:03.624293 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:05:03.624293 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:05:03.625545 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:05:03.625545 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:05:03.625545 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 09:05:03.625545 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 09:05:03.625545 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 09:05:03.625545 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 09:05:03.625545 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 09:05:03.625545 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 09:05:03.625545 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 14 09:05:03.625545 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 14 09:05:03.626286 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:03.627483 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 320/650] 10.222.0.26 () {56 vars in 1099 bytes} [Thu Jul 14 09:05:03 2022] GET /identity/v3/users/cinder => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:05:03.628430 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1d3770-0a9d-447e-af23-abc40e378cbc None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:03.661726 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a4e22d-1705-4ec2-883a-ae6207d27b11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:03.662526 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a4e22d-1705-4ec2-883a-ae6207d27b11 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:03.662802 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a4e22d-1705-4ec2-883a-ae6207d27b11 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:03.663091 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a4e22d-1705-4ec2-883a-ae6207d27b11 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:03.674927 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a4e22d-1705-4ec2-883a-ae6207d27b11 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:03.677133 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a4e22d-1705-4ec2-883a-ae6207d27b11 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:03.699360 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-f7a4e22d-1705-4ec2-883a-ae6207d27b11 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 09:05:03.699360 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:05:03.699360 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:05:03.699360 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:05:03.699360 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:05:03.699360 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:05:03.699360 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:05:03.699360 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:05:03.699360 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:05:03.699360 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:05:03.699360 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:05:03.699360 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:05:03.699360 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 09:05:03.699360 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 09:05:03.699360 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 09:05:03.700297 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 09:05:03.700297 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:05:03.700297 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:05:03.700297 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:05:03.700297 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:05:03.700297 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:05:03.700297 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:05:03.700297 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:05:03.700297 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 14 09:05:03.700297 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:05:03.700297 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:05:03.700297 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:05:03.700297 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:05:03.700297 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:05:03.700297 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:05:03.700297 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:05:03.700297 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:05:03.701224 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 09:05:03.701224 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 09:05:03.701224 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 09:05:03.701224 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 14 09:05:03.705933 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 331/651] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jul 14 09:05:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:05:03.706524 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a4e22d-1705-4ec2-883a-ae6207d27b11 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:03.730032 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1d3770-0a9d-447e-af23-abc40e378cbc None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:03.768063 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebb46f1-0f1b-4b1f-9e0a-fa8acf822b4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:03.769021 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebb46f1-0f1b-4b1f-9e0a-fa8acf822b4e None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:03.769303 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebb46f1-0f1b-4b1f-9e0a-fa8acf822b4e None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:03.769576 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebb46f1-0f1b-4b1f-9e0a-fa8acf822b4e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:03.776537 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebb46f1-0f1b-4b1f-9e0a-fa8acf822b4e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:03.777979 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebb46f1-0f1b-4b1f-9e0a-fa8acf822b4e None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:03.793498 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-6ebb46f1-0f1b-4b1f-9e0a-fa8acf822b4e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 09:05:03.793498 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:05:03.793498 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:05:03.793498 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:05:03.793498 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:05:03.793498 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:05:03.793498 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:05:03.793498 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:05:03.793498 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:05:03.793498 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:05:03.793498 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:05:03.793498 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:05:03.793498 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 09:05:03.793498 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 09:05:03.793498 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 09:05:03.794437 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 09:05:03.794437 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:05:03.794437 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:05:03.794437 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:05:03.794437 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:05:03.794437 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:05:03.794437 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:05:03.794437 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:05:03.794437 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 14 09:05:03.794437 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:05:03.794437 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:05:03.794437 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:05:03.794437 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:05:03.794437 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:05:03.794437 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:05:03.794437 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:05:03.794437 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:05:03.795483 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 09:05:03.795483 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 09:05:03.795483 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 09:05:03.795483 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 09:05:03.795483 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 09:05:03.795483 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 09:05:03.795483 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 09:05:03.795483 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 14 09:05:03.795483 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 321/652] 10.222.0.26 () {56 vars in 1111 bytes} [Thu Jul 14 09:05:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:05:03.795992 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebb46f1-0f1b-4b1f-9e0a-fa8acf822b4e None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:03.817623 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a4e22d-1705-4ec2-883a-ae6207d27b11 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:03.831797 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:03.857070 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484bd5fd-5589-4638-883f-71333a3b9f2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:03.857899 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484bd5fd-5589-4638-883f-71333a3b9f2d None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:03.858184 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484bd5fd-5589-4638-883f-71333a3b9f2d None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:03.858456 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484bd5fd-5589-4638-883f-71333a3b9f2d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:03.884379 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:03.898007 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebb46f1-0f1b-4b1f-9e0a-fa8acf822b4e None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:03.929611 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484bd5fd-5589-4638-883f-71333a3b9f2d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:03.930555 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ce83ef-21fd-4a61-acf0-18b641f23c9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:03.931063 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484bd5fd-5589-4638-883f-71333a3b9f2d None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:03.931239 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ce83ef-21fd-4a61-acf0-18b641f23c9f None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:03.931548 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ce83ef-21fd-4a61-acf0-18b641f23c9f None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:03.931821 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ce83ef-21fd-4a61-acf0-18b641f23c9f None admin] PATH_INFO: `/v3/users` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:03.932653 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ce83ef-21fd-4a61-acf0-18b641f23c9f None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:05:03.932959 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ce83ef-21fd-4a61-acf0-18b641f23c9f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:03.934364 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ce83ef-21fd-4a61-acf0-18b641f23c9f None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:03.951794 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 322/653] 10.222.0.26 () {56 vars in 1137 bytes} [Thu Jul 14 09:05:03 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:05:03.952529 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ce83ef-21fd-4a61-acf0-18b641f23c9f None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:04.040214 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 332/654] 10.222.0.26 () {58 vars in 1282 bytes} [Thu Jul 14 09:05:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 09:05:04.041202 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484bd5fd-5589-4638-883f-71333a3b9f2d None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:04.056375 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:04.066509 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ce83ef-21fd-4a61-acf0-18b641f23c9f None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:04.099962 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:04.103865 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0a0921-79f9-4a89-87de-d1d243c9a8f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:04.104535 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0a0921-79f9-4a89-87de-d1d243c9a8f4 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:04.104835 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0a0921-79f9-4a89-87de-d1d243c9a8f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:04.105108 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0a0921-79f9-4a89-87de-d1d243c9a8f4 None admin] PATH_INFO: `/v3/domains` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:04.105831 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0a0921-79f9-4a89-87de-d1d243c9a8f4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:05:04.106144 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0a0921-79f9-4a89-87de-d1d243c9a8f4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:04.107548 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0a0921-79f9-4a89-87de-d1d243c9a8f4 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:04.118932 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 323/655] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Jul 14 09:05:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:05:04.120187 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c07ef0-6128-4e00-b078-e1bd33f8279e None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:04.120525 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c07ef0-6128-4e00-b078-e1bd33f8279e None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:04.120809 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c07ef0-6128-4e00-b078-e1bd33f8279e None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:04.122193 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 324/656] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:05:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:05:04.122722 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c07ef0-6128-4e00-b078-e1bd33f8279e None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:04.150886 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484bd5fd-5589-4638-883f-71333a3b9f2d None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:04.184710 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12718061-6ea8-4f3b-bf91-fb766c4c3f08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:04.185374 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12718061-6ea8-4f3b-bf91-fb766c4c3f08 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:04.185655 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12718061-6ea8-4f3b-bf91-fb766c4c3f08 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:04.185925 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12718061-6ea8-4f3b-bf91-fb766c4c3f08 None admin] PATH_INFO: `/v3/projects` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:04.186716 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12718061-6ea8-4f3b-bf91-fb766c4c3f08 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:05:04.187013 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12718061-6ea8-4f3b-bf91-fb766c4c3f08 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:04.188525 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12718061-6ea8-4f3b-bf91-fb766c4c3f08 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:04.200090 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 333/657] 10.222.0.26 () {56 vars in 1148 bytes} [Thu Jul 14 09:05:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:05:04.200844 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12718061-6ea8-4f3b-bf91-fb766c4c3f08 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:04.223050 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c07ef0-6128-4e00-b078-e1bd33f8279e None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:04.256425 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1f37c7-b44c-44bd-83af-309a19f4cfcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:04.257070 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1f37c7-b44c-44bd-83af-309a19f4cfcd None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:04.257357 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1f37c7-b44c-44bd-83af-309a19f4cfcd None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:04.257631 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1f37c7-b44c-44bd-83af-309a19f4cfcd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:04.278170 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:04.299597 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12718061-6ea8-4f3b-bf91-fb766c4c3f08 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:04.315241 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:04.324851 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1f37c7-b44c-44bd-83af-309a19f4cfcd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:04.326303 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1f37c7-b44c-44bd-83af-309a19f4cfcd None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:04.332323 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59b5a54-5caa-4c12-b9f5-367add727e19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:04.333115 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59b5a54-5caa-4c12-b9f5-367add727e19 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:04.333393 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59b5a54-5caa-4c12-b9f5-367add727e19 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:04.333670 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59b5a54-5caa-4c12-b9f5-367add727e19 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:04.344846 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59b5a54-5caa-4c12-b9f5-367add727e19 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:04.346281 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59b5a54-5caa-4c12-b9f5-367add727e19 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:04.363112 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-a59b5a54-5caa-4c12-b9f5-367add727e19 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 09:05:04.363112 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:05:04.363112 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:05:04.363112 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:05:04.363112 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:05:04.363112 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:05:04.363112 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:05:04.363112 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:05:04.363112 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:05:04.363112 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:05:04.363112 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:05:04.363112 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:05:04.363112 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 09:05:04.363112 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 09:05:04.363112 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 09:05:04.364234 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 09:05:04.364234 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:05:04.364234 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:05:04.364234 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:05:04.364234 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:05:04.364234 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:05:04.364234 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:05:04.364234 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:05:04.364234 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 14 09:05:04.364234 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:05:04.364234 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:05:04.364234 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:05:04.364234 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:05:04.364234 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:05:04.364234 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:05:04.364234 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:05:04.364234 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:05:04.365415 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 09:05:04.365415 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 09:05:04.365415 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 09:05:04.365415 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 14 09:05:04.365415 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 334/658] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jul 14 09:05:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:05:04.365790 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59b5a54-5caa-4c12-b9f5-367add727e19 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:04.429020 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 325/659] 10.222.0.26 () {58 vars in 1282 bytes} [Thu Jul 14 09:05:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 09:05:04.430432 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eabf812-97d1-468c-806b-581901cc5c0e None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:04.430720 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eabf812-97d1-468c-806b-581901cc5c0e None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:04.430995 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eabf812-97d1-468c-806b-581901cc5c0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:04.446756 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-0eabf812-97d1-468c-806b-581901cc5c0e None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:05:04.448081 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-0eabf812-97d1-468c-806b-581901cc5c0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:05:04.465001 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59b5a54-5caa-4c12-b9f5-367add727e19 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:04.493285 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:04.498894 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ed4df5-1f72-46b6-80f6-81491b6a1891 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:04.499721 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ed4df5-1f72-46b6-80f6-81491b6a1891 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:04.500003 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ed4df5-1f72-46b6-80f6-81491b6a1891 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:04.500278 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ed4df5-1f72-46b6-80f6-81491b6a1891 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:04.507731 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ed4df5-1f72-46b6-80f6-81491b6a1891 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:04.509180 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ed4df5-1f72-46b6-80f6-81491b6a1891 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:04.514122 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 326/660] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:05:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 09:05:04.514939 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eabf812-97d1-468c-806b-581901cc5c0e None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:04.523099 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-d7ed4df5-1f72-46b6-80f6-81491b6a1891 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 14 09:05:04.523099 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:05:04.523099 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:05:04.523099 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:05:04.523099 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:05:04.523099 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:05:04.523099 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:05:04.523099 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:05:04.523099 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:05:04.523099 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:05:04.523099 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:05:04.523099 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:05:04.523099 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 09:05:04.523099 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 09:05:04.523099 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 09:05:04.523099 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 09:05:04.524072 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:05:04.524072 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:05:04.524072 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 09:05:04.524072 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 09:05:04.524072 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 09:05:04.524072 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 09:05:04.524072 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:05:04.524072 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:05:04.524072 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:05:04.524072 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:05:04.524072 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:05:04.524072 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 14 09:05:04.524072 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:05:04.524072 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:05:04.524072 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:05:04.524072 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:05:04.524072 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:05:04.524072 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:05:04.525133 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:05:04.525133 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:05:04.525133 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 09:05:04.525133 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 09:05:04.525133 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 09:05:04.525133 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 09:05:04.525133 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 09:05:04.525133 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 09:05:04.525133 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 14 09:05:04.525133 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 14 09:05:04.525133 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 335/661] 10.222.0.26 () {56 vars in 1102 bytes} [Thu Jul 14 09:05:04 2022] GET /identity/v3/users/neutron => generated 85 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:05:04.525133 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ed4df5-1f72-46b6-80f6-81491b6a1891 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:04.558183 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:04.630296 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ed4df5-1f72-46b6-80f6-81491b6a1891 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:04.634321 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eabf812-97d1-468c-806b-581901cc5c0e None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:04.664675 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7737872-2d8a-4902-bb6f-d30d0ccf653f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:04.665337 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7737872-2d8a-4902-bb6f-d30d0ccf653f None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:04.665644 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7737872-2d8a-4902-bb6f-d30d0ccf653f None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:04.665948 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7737872-2d8a-4902-bb6f-d30d0ccf653f None admin] PATH_INFO: `/v3/domains` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:04.666678 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7737872-2d8a-4902-bb6f-d30d0ccf653f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:05:04.666985 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7737872-2d8a-4902-bb6f-d30d0ccf653f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:04.668498 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7737872-2d8a-4902-bb6f-d30d0ccf653f None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:04.668877 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a500f3d5-1cd3-4528-b7ff-aca2f1039bb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:04.669532 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a500f3d5-1cd3-4528-b7ff-aca2f1039bb0 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:04.669818 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a500f3d5-1cd3-4528-b7ff-aca2f1039bb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:04.670087 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a500f3d5-1cd3-4528-b7ff-aca2f1039bb0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:04.670924 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a500f3d5-1cd3-4528-b7ff-aca2f1039bb0 None admin] RBAC: Adding query filter params (role.id=f5c487bc605f45df82ef6367c80fae70, scope.project.id=33bf210d015847439655b9943d1b0261, user.id=538fff3c5a2b487eb44ae7c1a1198102) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:05:04.671275 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a500f3d5-1cd3-4528-b7ff-aca2f1039bb0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:04.672693 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a500f3d5-1cd3-4528-b7ff-aca2f1039bb0 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:04.679092 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 327/662] 10.222.0.26 () {56 vars in 1374 bytes} [Thu Jul 14 09:05:04 2022] GET /identity/v3/role_assignments?user.id=538fff3c5a2b487eb44ae7c1a1198102&scope.project.id=33bf210d015847439655b9943d1b0261&role.id=f5c487bc605f45df82ef6367c80fae70 => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:05:04.680002 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a500f3d5-1cd3-4528-b7ff-aca2f1039bb0 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:04.684639 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 336/663] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Jul 14 09:05:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:05:04.684639 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7737872-2d8a-4902-bb6f-d30d0ccf653f None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:04.747763 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:04.781897 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:04.796978 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7737872-2d8a-4902-bb6f-d30d0ccf653f None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:04.813749 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a500f3d5-1cd3-4528-b7ff-aca2f1039bb0 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:04.839597 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9717993-7749-432e-9ff4-ae759524a277 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:04.840224 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9717993-7749-432e-9ff4-ae759524a277 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:04.840700 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9717993-7749-432e-9ff4-ae759524a277 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:04.841200 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9717993-7749-432e-9ff4-ae759524a277 None admin] PATH_INFO: `/v3/users` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:04.842033 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9717993-7749-432e-9ff4-ae759524a277 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:05:04.842341 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9717993-7749-432e-9ff4-ae759524a277 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:04.845053 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9717993-7749-432e-9ff4-ae759524a277 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:04.858817 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af041cf-f093-4f9b-8d94-960ce040f140 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:04.859761 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af041cf-f093-4f9b-8d94-960ce040f140 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:04.860376 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af041cf-f093-4f9b-8d94-960ce040f140 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:04.860376 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af041cf-f093-4f9b-8d94-960ce040f140 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:04.868223 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 337/664] 10.222.0.26 () {56 vars in 1139 bytes} [Thu Jul 14 09:05:04 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:05:04.869811 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9717993-7749-432e-9ff4-ae759524a277 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:04.873396 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af041cf-f093-4f9b-8d94-960ce040f140 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:04.875921 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af041cf-f093-4f9b-8d94-960ce040f140 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:04.897249 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-1af041cf-f093-4f9b-8d94-960ce040f140 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 09:05:04.897249 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:05:04.897249 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:05:04.897249 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:05:04.897249 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:05:04.897249 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:05:04.897249 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:05:04.897249 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:05:04.897249 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:05:04.897249 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:05:04.897249 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:05:04.897249 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:05:04.897249 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 09:05:04.897249 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 09:05:04.897249 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 09:05:04.898214 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 09:05:04.898214 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:05:04.898214 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:05:04.898214 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:05:04.898214 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:05:04.898214 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:05:04.898214 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:05:04.898214 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:05:04.898214 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 14 09:05:04.898214 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:05:04.898214 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:05:04.898214 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:05:04.898214 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:05:04.898214 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:05:04.898214 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:05:04.898214 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:05:04.898214 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:05:04.899133 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 09:05:04.899133 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 09:05:04.899133 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 09:05:04.899133 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 14 09:05:04.899133 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 328/665] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jul 14 09:05:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:05:04.899490 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af041cf-f093-4f9b-8d94-960ce040f140 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:04.979624 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9717993-7749-432e-9ff4-ae759524a277 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:05.004414 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:05.007955 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af041cf-f093-4f9b-8d94-960ce040f140 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:05.021157 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34aa0823-ddeb-4860-b18a-16959a286c1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:05.021157 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34aa0823-ddeb-4860-b18a-16959a286c1a None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:05.021157 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34aa0823-ddeb-4860-b18a-16959a286c1a None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:05.021554 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34aa0823-ddeb-4860-b18a-16959a286c1a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:05.032608 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34aa0823-ddeb-4860-b18a-16959a286c1a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 09:05:05.040735 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-34aa0823-ddeb-4860-b18a-16959a286c1a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 09:05:05.040735 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:05:05.040735 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:05:05.040735 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:05:05.040735 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:05:05.040735 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:05:05.040735 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:05:05.040735 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:05:05.040735 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:05:05.040735 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:05:05.040735 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:05:05.040735 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:05:05.040735 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 09:05:05.040735 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 09:05:05.040735 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 09:05:05.040735 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 09:05:05.041746 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 09:05:05.041746 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 09:05:05.041746 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 09:05:05.041746 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 09:05:05.041746 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:05:05.041746 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:05:05.041746 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:05:05.041746 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:05:05.041746 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:05:05.041746 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:05:05.041746 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:05:05.041746 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 14 09:05:05.041746 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:05:05.041746 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:05:05.041746 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:05:05.041746 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:05:05.042619 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:05:05.042619 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:05:05.042619 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:05:05.042619 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:05:05.042619 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 09:05:05.042619 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 09:05:05.042619 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 09:05:05.042619 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 09:05:05.042619 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 09:05:05.042619 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 09:05:05.042619 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 09:05:05.042619 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 14 09:05:05.042619 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 338/666] 10.222.0.26 () {56 vars in 1102 bytes} [Thu Jul 14 09:05:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:05:05.043374 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34aa0823-ddeb-4860-b18a-16959a286c1a None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:05.051863 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d61d4b7-7980-4e63-850e-cbddc558c6b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:05.052651 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d61d4b7-7980-4e63-850e-cbddc558c6b0 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:05.052940 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d61d4b7-7980-4e63-850e-cbddc558c6b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:05.053116 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:05.053229 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d61d4b7-7980-4e63-850e-cbddc558c6b0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:05.060866 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d61d4b7-7980-4e63-850e-cbddc558c6b0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:05.062319 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d61d4b7-7980-4e63-850e-cbddc558c6b0 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:05.076843 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-3d61d4b7-7980-4e63-850e-cbddc558c6b0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 09:05:05.076843 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:05:05.076843 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:05:05.076843 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:05:05.076843 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:05:05.076843 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:05:05.076843 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:05:05.076843 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:05:05.076843 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:05:05.076843 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:05:05.076843 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:05:05.076843 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:05:05.076843 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 09:05:05.076843 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 09:05:05.076843 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 09:05:05.077758 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 09:05:05.077758 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:05:05.077758 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:05:05.077758 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:05:05.077758 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:05:05.077758 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:05:05.077758 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:05:05.077758 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:05:05.077758 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 14 09:05:05.077758 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:05:05.077758 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:05:05.077758 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:05:05.077758 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:05:05.077758 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:05:05.077758 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:05:05.077758 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:05:05.077758 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:05:05.078651 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 09:05:05.078651 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 09:05:05.078651 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 09:05:05.078651 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 09:05:05.078651 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 09:05:05.078651 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 09:05:05.078651 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 09:05:05.078651 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 14 09:05:05.078651 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 329/667] 10.222.0.26 () {56 vars in 1111 bytes} [Thu Jul 14 09:05:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:05:05.079178 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d61d4b7-7980-4e63-850e-cbddc558c6b0 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:05.154107 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34aa0823-ddeb-4860-b18a-16959a286c1a None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:05.185618 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d61d4b7-7980-4e63-850e-cbddc558c6b0 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:05.188226 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a243a587-ec4b-4d4f-be25-cfd60f43051d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:05.188974 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a243a587-ec4b-4d4f-be25-cfd60f43051d None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:05.189258 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a243a587-ec4b-4d4f-be25-cfd60f43051d None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:05.189531 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a243a587-ec4b-4d4f-be25-cfd60f43051d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:05.223594 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdef5b2-f9b2-49b8-a4a8-4277cd68d7f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:05.224262 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdef5b2-f9b2-49b8-a4a8-4277cd68d7f9 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:05.224544 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdef5b2-f9b2-49b8-a4a8-4277cd68d7f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:05.224821 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdef5b2-f9b2-49b8-a4a8-4277cd68d7f9 None admin] PATH_INFO: `/v3/domains` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:05.225549 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdef5b2-f9b2-49b8-a4a8-4277cd68d7f9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:05:05.225857 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdef5b2-f9b2-49b8-a4a8-4277cd68d7f9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:05.227260 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdef5b2-f9b2-49b8-a4a8-4277cd68d7f9 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:05.239325 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 330/668] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Jul 14 09:05:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:05:05.240395 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdef5b2-f9b2-49b8-a4a8-4277cd68d7f9 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:05.263129 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:05.267243 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a243a587-ec4b-4d4f-be25-cfd60f43051d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:05.268923 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a243a587-ec4b-4d4f-be25-cfd60f43051d None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:05.278175 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:05.350593 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdef5b2-f9b2-49b8-a4a8-4277cd68d7f9 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:05.377620 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 339/669] 10.222.0.26 () {58 vars in 1282 bytes} [Thu Jul 14 09:05:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 09:05:05.377620 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a243a587-ec4b-4d4f-be25-cfd60f43051d None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:05.390737 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70715c3a-da39-4e92-aac5-d4dff6c77678 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:05.391464 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70715c3a-da39-4e92-aac5-d4dff6c77678 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:05.391746 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70715c3a-da39-4e92-aac5-d4dff6c77678 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:05.392014 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70715c3a-da39-4e92-aac5-d4dff6c77678 None admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:05.392763 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70715c3a-da39-4e92-aac5-d4dff6c77678 None admin] RBAC: Adding query filter params (name=service) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:05:05.393066 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70715c3a-da39-4e92-aac5-d4dff6c77678 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:05.394428 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70715c3a-da39-4e92-aac5-d4dff6c77678 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:05.402791 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 331/670] 10.222.0.26 () {56 vars in 1103 bytes} [Thu Jul 14 09:05:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:05:05.404474 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70715c3a-da39-4e92-aac5-d4dff6c77678 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:05.484795 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a243a587-ec4b-4d4f-be25-cfd60f43051d None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:05.487872 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:05.494561 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:05.507432 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70715c3a-da39-4e92-aac5-d4dff6c77678 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:05.523463 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0983e0a-357d-4751-9692-270786d19a3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:05.524182 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0983e0a-357d-4751-9692-270786d19a3a None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:05.524467 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0983e0a-357d-4751-9692-270786d19a3a None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:05.524764 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0983e0a-357d-4751-9692-270786d19a3a None admin] PATH_INFO: `/v3/projects` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:05.525554 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0983e0a-357d-4751-9692-270786d19a3a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:05:05.525865 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0983e0a-357d-4751-9692-270786d19a3a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:05.527454 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0983e0a-357d-4751-9692-270786d19a3a None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:05.541162 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 340/671] 10.222.0.26 () {56 vars in 1148 bytes} [Thu Jul 14 09:05:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:05:05.542104 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0983e0a-357d-4751-9692-270786d19a3a None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:05.545638 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa84bee-cd78-4fc5-bf74-4a1847629d35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:05.547670 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa84bee-cd78-4fc5-bf74-4a1847629d35 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:05.548143 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa84bee-cd78-4fc5-bf74-4a1847629d35 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:05.548595 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa84bee-cd78-4fc5-bf74-4a1847629d35 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:05.558519 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa84bee-cd78-4fc5-bf74-4a1847629d35 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:05.560127 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa84bee-cd78-4fc5-bf74-4a1847629d35 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:05.574534 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-7aa84bee-cd78-4fc5-bf74-4a1847629d35 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 09:05:05.574534 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:05:05.574534 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:05:05.574534 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:05:05.574534 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:05:05.574534 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:05:05.574534 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:05:05.574534 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:05:05.574534 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:05:05.574534 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:05:05.574534 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:05:05.574534 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:05:05.574534 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 09:05:05.574534 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 09:05:05.574534 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 09:05:05.575629 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 09:05:05.575629 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:05:05.575629 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:05:05.575629 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:05:05.575629 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:05:05.575629 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:05:05.575629 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:05:05.575629 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:05:05.575629 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 14 09:05:05.575629 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:05:05.575629 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:05:05.575629 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:05:05.575629 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:05:05.575629 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:05:05.575629 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:05:05.575629 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:05:05.575629 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:05:05.576757 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 09:05:05.576757 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 09:05:05.576757 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 09:05:05.576757 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 09:05:05.576757 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 09:05:05.576757 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 09:05:05.576757 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 09:05:05.576757 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 14 09:05:05.580763 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 332/672] 10.222.0.26 () {56 vars in 1111 bytes} [Thu Jul 14 09:05:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:05:05.581630 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa84bee-cd78-4fc5-bf74-4a1847629d35 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:05.652594 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0983e0a-357d-4751-9692-270786d19a3a None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:05.685823 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa84bee-cd78-4fc5-bf74-4a1847629d35 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:05.688708 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0224ec0d-f531-4c72-9a8c-dd36a87ebb9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:05.689575 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0224ec0d-f531-4c72-9a8c-dd36a87ebb9a None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:05.689854 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0224ec0d-f531-4c72-9a8c-dd36a87ebb9a None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:05.690123 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0224ec0d-f531-4c72-9a8c-dd36a87ebb9a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:05.701835 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0224ec0d-f531-4c72-9a8c-dd36a87ebb9a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:05.703546 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0224ec0d-f531-4c72-9a8c-dd36a87ebb9a None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:05.721309 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-0224ec0d-f531-4c72-9a8c-dd36a87ebb9a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 09:05:05.721309 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:05:05.721309 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:05:05.721309 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:05:05.721309 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:05:05.721309 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:05:05.721309 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:05:05.721309 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:05:05.721309 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:05:05.721309 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:05:05.721309 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:05:05.721309 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:05:05.721309 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 09:05:05.721309 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 09:05:05.721309 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 09:05:05.722386 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 09:05:05.722386 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:05:05.722386 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:05:05.722386 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:05:05.722386 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:05:05.722386 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:05:05.722386 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:05:05.722386 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:05:05.722386 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 14 09:05:05.722386 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:05:05.722386 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:05:05.722386 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:05:05.722386 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:05:05.722386 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:05:05.722386 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:05:05.722386 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:05:05.722386 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:05:05.723357 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 09:05:05.723357 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 09:05:05.723357 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 09:05:05.723357 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 14 09:05:05.723357 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 341/673] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jul 14 09:05:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:05:05.723357 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0224ec0d-f531-4c72-9a8c-dd36a87ebb9a None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:05.724537 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6efc09-fd62-48ed-a885-e80a9dde3df1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:05.725325 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6efc09-fd62-48ed-a885-e80a9dde3df1 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:05.725606 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6efc09-fd62-48ed-a885-e80a9dde3df1 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:05.725875 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6efc09-fd62-48ed-a885-e80a9dde3df1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:05.735622 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:05.737832 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6efc09-fd62-48ed-a885-e80a9dde3df1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:05.739775 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6efc09-fd62-48ed-a885-e80a9dde3df1 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:05.749001 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:05.758197 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-5b6efc09-fd62-48ed-a885-e80a9dde3df1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 09:05:05.758197 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:05:05.758197 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:05:05.758197 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:05:05.758197 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:05:05.758197 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:05:05.758197 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:05:05.758197 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:05:05.758197 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:05:05.758197 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:05:05.758197 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:05:05.758197 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:05:05.758197 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 09:05:05.758197 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 09:05:05.758197 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 09:05:05.759368 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 09:05:05.759368 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:05:05.759368 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:05:05.759368 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:05:05.759368 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:05:05.759368 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:05:05.759368 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:05:05.759368 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:05:05.759368 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 14 09:05:05.759368 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:05:05.759368 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:05:05.759368 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:05:05.759368 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:05:05.759368 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:05:05.759368 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:05:05.759368 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:05:05.759368 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:05:05.760411 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 09:05:05.760411 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 09:05:05.760411 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 09:05:05.760411 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 14 09:05:05.760411 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 333/674] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jul 14 09:05:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:05:05.760768 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6efc09-fd62-48ed-a885-e80a9dde3df1 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:05.843960 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0224ec0d-f531-4c72-9a8c-dd36a87ebb9a None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:05.869051 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6efc09-fd62-48ed-a885-e80a9dde3df1 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:05.883843 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45f111f-7921-4981-83ad-4da5dc38b73c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:05.884654 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45f111f-7921-4981-83ad-4da5dc38b73c None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:05.884939 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45f111f-7921-4981-83ad-4da5dc38b73c None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:05.885249 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45f111f-7921-4981-83ad-4da5dc38b73c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:05.886089 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45f111f-7921-4981-83ad-4da5dc38b73c None admin] RBAC: Adding query filter params (role.id=f5c487bc605f45df82ef6367c80fae70, scope.project.id=33bf210d015847439655b9943d1b0261, user.id=872dbb91c9e54f13b1a31fc8242126df) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:05:05.886400 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45f111f-7921-4981-83ad-4da5dc38b73c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:05.887877 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45f111f-7921-4981-83ad-4da5dc38b73c None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:05.893676 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 342/675] 10.222.0.26 () {56 vars in 1374 bytes} [Thu Jul 14 09:05:05 2022] GET /identity/v3/role_assignments?user.id=872dbb91c9e54f13b1a31fc8242126df&scope.project.id=33bf210d015847439655b9943d1b0261&role.id=f5c487bc605f45df82ef6367c80fae70 => generated 608 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:05:05.894453 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45f111f-7921-4981-83ad-4da5dc38b73c None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:05.927149 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855e60e3-d9b6-42d6-aeab-136e5e8382bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:05.927866 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855e60e3-d9b6-42d6-aeab-136e5e8382bc None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:05.928148 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855e60e3-d9b6-42d6-aeab-136e5e8382bc None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:05.928424 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855e60e3-d9b6-42d6-aeab-136e5e8382bc None admin] PATH_INFO: `/v3/projects` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:05.929238 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855e60e3-d9b6-42d6-aeab-136e5e8382bc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:05:05.929548 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855e60e3-d9b6-42d6-aeab-136e5e8382bc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:05.931138 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855e60e3-d9b6-42d6-aeab-136e5e8382bc None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:05.945746 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 334/676] 10.222.0.26 () {56 vars in 1148 bytes} [Thu Jul 14 09:05:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:05:05.946684 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855e60e3-d9b6-42d6-aeab-136e5e8382bc None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:06.009539 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:06.015862 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:06.038895 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45f111f-7921-4981-83ad-4da5dc38b73c None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:06.068431 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855e60e3-d9b6-42d6-aeab-136e5e8382bc None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:06.088647 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c72df2-c776-4f26-ab48-65c267400565 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:06.089344 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c72df2-c776-4f26-ab48-65c267400565 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:06.089716 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c72df2-c776-4f26-ab48-65c267400565 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:06.090004 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c72df2-c776-4f26-ab48-65c267400565 None admin] PATH_INFO: `/v3/domains` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:06.091220 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c72df2-c776-4f26-ab48-65c267400565 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:05:06.091220 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c72df2-c776-4f26-ab48-65c267400565 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:06.092433 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c72df2-c776-4f26-ab48-65c267400565 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:06.106515 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 343/677] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Jul 14 09:05:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:05:06.107173 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c72df2-c776-4f26-ab48-65c267400565 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:06.117101 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eae8d3f-4ec6-4992-be53-45708c324e3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:06.118067 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eae8d3f-4ec6-4992-be53-45708c324e3e None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:06.118604 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eae8d3f-4ec6-4992-be53-45708c324e3e None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:06.118890 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eae8d3f-4ec6-4992-be53-45708c324e3e None admin] PATH_INFO: `/v3/domains` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:06.119671 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eae8d3f-4ec6-4992-be53-45708c324e3e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:05:06.119979 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eae8d3f-4ec6-4992-be53-45708c324e3e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:06.121358 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eae8d3f-4ec6-4992-be53-45708c324e3e None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:06.134653 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 335/678] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Jul 14 09:05:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:05:06.135518 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eae8d3f-4ec6-4992-be53-45708c324e3e None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:06.243596 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c72df2-c776-4f26-ab48-65c267400565 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:06.262543 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eae8d3f-4ec6-4992-be53-45708c324e3e None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:06.276516 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:06.297305 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988fc440-f056-4559-8fac-29471dab56c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:06.298213 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988fc440-f056-4559-8fac-29471dab56c0 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:06.298667 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988fc440-f056-4559-8fac-29471dab56c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:06.299091 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988fc440-f056-4559-8fac-29471dab56c0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:06.300095 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988fc440-f056-4559-8fac-29471dab56c0 None admin] RBAC: Adding query filter params (role.id=f5c487bc605f45df82ef6367c80fae70, scope.project.id=33bf210d015847439655b9943d1b0261, user.id=e76f1952a5b146808844b9f2b7395035) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:05:06.300552 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988fc440-f056-4559-8fac-29471dab56c0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:06.301232 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc0d1f3-3cb5-44da-9fa0-a8d0c3e781d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:06.302127 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc0d1f3-3cb5-44da-9fa0-a8d0c3e781d9 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:06.302478 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988fc440-f056-4559-8fac-29471dab56c0 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:06.302591 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc0d1f3-3cb5-44da-9fa0-a8d0c3e781d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:06.302937 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc0d1f3-3cb5-44da-9fa0-a8d0c3e781d9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:06.310435 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc0d1f3-3cb5-44da-9fa0-a8d0c3e781d9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:06.310655 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 344/679] 10.222.0.26 () {56 vars in 1374 bytes} [Thu Jul 14 09:05:06 2022] GET /identity/v3/role_assignments?user.id=e76f1952a5b146808844b9f2b7395035&scope.project.id=33bf210d015847439655b9943d1b0261&role.id=f5c487bc605f45df82ef6367c80fae70 => generated 608 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:05:06.311623 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988fc440-f056-4559-8fac-29471dab56c0 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:06.312415 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc0d1f3-3cb5-44da-9fa0-a8d0c3e781d9 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:06.322254 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:06.329223 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-0fc0d1f3-3cb5-44da-9fa0-a8d0c3e781d9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 09:05:06.329223 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:05:06.329223 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:05:06.329223 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:05:06.329223 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:05:06.329223 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:05:06.329223 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:05:06.329223 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:05:06.329223 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:05:06.329223 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:05:06.329223 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:05:06.329223 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:05:06.329223 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 09:05:06.329223 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 09:05:06.329223 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 09:05:06.330394 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 09:05:06.330394 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:05:06.330394 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:05:06.330394 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:05:06.330394 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:05:06.330394 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:05:06.330394 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:05:06.330394 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:05:06.330394 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 14 09:05:06.330394 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:05:06.330394 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:05:06.330394 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:05:06.330394 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:05:06.330394 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:05:06.330394 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:05:06.330394 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:05:06.330394 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:05:06.331562 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 09:05:06.331562 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 09:05:06.331562 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 09:05:06.331562 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 09:05:06.331562 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 09:05:06.331562 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 09:05:06.331562 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 09:05:06.331562 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 14 09:05:06.337895 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 336/680] 10.222.0.26 () {56 vars in 1111 bytes} [Thu Jul 14 09:05:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:05:06.339337 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc0d1f3-3cb5-44da-9fa0-a8d0c3e781d9 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:06.437808 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988fc440-f056-4559-8fac-29471dab56c0 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:06.482768 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03362c7-fa4d-4745-ba9c-8cd4ce5a7818 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:06.483783 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03362c7-fa4d-4745-ba9c-8cd4ce5a7818 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:06.484163 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03362c7-fa4d-4745-ba9c-8cd4ce5a7818 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:06.484534 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03362c7-fa4d-4745-ba9c-8cd4ce5a7818 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:06.500805 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03362c7-fa4d-4745-ba9c-8cd4ce5a7818 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:06.502287 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03362c7-fa4d-4745-ba9c-8cd4ce5a7818 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:06.523108 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-b03362c7-fa4d-4745-ba9c-8cd4ce5a7818 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 14 09:05:06.523108 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:05:06.523108 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:05:06.523108 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:05:06.523108 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:05:06.523108 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:05:06.523108 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:05:06.523108 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:05:06.523108 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:05:06.523108 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:05:06.523108 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:05:06.523108 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:05:06.523108 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 09:05:06.523108 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 09:05:06.523108 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 09:05:06.523108 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 09:05:06.524324 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:05:06.524324 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:05:06.524324 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 09:05:06.524324 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 09:05:06.524324 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 09:05:06.524324 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 09:05:06.524324 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:05:06.524324 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:05:06.524324 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:05:06.524324 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:05:06.524324 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:05:06.524324 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 14 09:05:06.524324 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:05:06.524324 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:05:06.524324 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:05:06.524324 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:05:06.524324 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:05:06.524324 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:05:06.525558 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:05:06.525558 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:05:06.525558 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 09:05:06.525558 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 09:05:06.525558 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 09:05:06.525558 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 09:05:06.525558 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 09:05:06.525558 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 09:05:06.525558 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 14 09:05:06.525558 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 14 09:05:06.525558 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 345/681] 10.222.0.26 () {56 vars in 1096 bytes} [Thu Jul 14 09:05:06 2022] GET /identity/v3/users/swift => generated 83 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:05:06.532340 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03362c7-fa4d-4745-ba9c-8cd4ce5a7818 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:06.546659 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc0d1f3-3cb5-44da-9fa0-a8d0c3e781d9 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:06.583590 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92aaffdd-d25c-439a-bc3b-afe2a4233bee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:06.584619 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92aaffdd-d25c-439a-bc3b-afe2a4233bee None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:06.585162 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92aaffdd-d25c-439a-bc3b-afe2a4233bee None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:06.585612 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92aaffdd-d25c-439a-bc3b-afe2a4233bee None admin] PATH_INFO: `/v3/projects` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:06.586613 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92aaffdd-d25c-439a-bc3b-afe2a4233bee None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:05:06.587253 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92aaffdd-d25c-439a-bc3b-afe2a4233bee None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:06.588978 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92aaffdd-d25c-439a-bc3b-afe2a4233bee None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:06.606842 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 337/682] 10.222.0.26 () {56 vars in 1148 bytes} [Thu Jul 14 09:05:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:05:06.608556 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92aaffdd-d25c-439a-bc3b-afe2a4233bee None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:06.623649 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:06.644632 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:06.660357 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03362c7-fa4d-4745-ba9c-8cd4ce5a7818 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:06.721141 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92aaffdd-d25c-439a-bc3b-afe2a4233bee None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:06.735896 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e13cca-ec15-4faf-980b-08913d85eeec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:06.736672 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e13cca-ec15-4faf-980b-08913d85eeec None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:06.737021 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e13cca-ec15-4faf-980b-08913d85eeec None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:06.737362 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e13cca-ec15-4faf-980b-08913d85eeec None admin] PATH_INFO: `/v3/users` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:06.738285 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e13cca-ec15-4faf-980b-08913d85eeec None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:05:06.738659 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e13cca-ec15-4faf-980b-08913d85eeec None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:06.740191 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e13cca-ec15-4faf-980b-08913d85eeec None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:06.758195 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06f5c9f-46c2-44d3-ac89-a143f5e55400 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:06.759132 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06f5c9f-46c2-44d3-ac89-a143f5e55400 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:06.759710 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06f5c9f-46c2-44d3-ac89-a143f5e55400 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:06.760129 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06f5c9f-46c2-44d3-ac89-a143f5e55400 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:06.761180 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06f5c9f-46c2-44d3-ac89-a143f5e55400 None admin] RBAC: Adding query filter params (role.id=f5c487bc605f45df82ef6367c80fae70, scope.project.id=33bf210d015847439655b9943d1b0261, user.id=952911c492444ffea4b211a24b564117) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:05:06.761696 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06f5c9f-46c2-44d3-ac89-a143f5e55400 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:06.763352 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06f5c9f-46c2-44d3-ac89-a143f5e55400 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:06.774966 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 346/684] 10.222.0.26 () {56 vars in 1135 bytes} [Thu Jul 14 09:05:06 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:05:06.775262 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 338/684] 10.222.0.26 () {56 vars in 1374 bytes} [Thu Jul 14 09:05:06 2022] GET /identity/v3/role_assignments?user.id=952911c492444ffea4b211a24b564117&scope.project.id=33bf210d015847439655b9943d1b0261&role.id=f5c487bc605f45df82ef6367c80fae70 => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 09:05:06.780539 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e13cca-ec15-4faf-980b-08913d85eeec None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:06.855682 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50e6eea-e463-464d-b592-0dc546658924 None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:06.856206 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50e6eea-e463-464d-b592-0dc546658924 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:06.856662 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50e6eea-e463-464d-b592-0dc546658924 None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:06.863235 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 339/685] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:05:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 09:05:06.864740 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30dafbc-bf04-4bc4-b4ef-ecb4101fa8de None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:06.865283 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30dafbc-bf04-4bc4-b4ef-ecb4101fa8de None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:06.865731 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30dafbc-bf04-4bc4-b4ef-ecb4101fa8de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:06.885197 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-e30dafbc-bf04-4bc4-b4ef-ecb4101fa8de None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:05:06.886491 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-e30dafbc-bf04-4bc4-b4ef-ecb4101fa8de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:05:06.933530 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e13cca-ec15-4faf-980b-08913d85eeec None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:06.958968 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:06.970621 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06736ab1-20ca-4a48-afb5-c2f50139da2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:06.971537 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06736ab1-20ca-4a48-afb5-c2f50139da2e None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:06.971821 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06736ab1-20ca-4a48-afb5-c2f50139da2e None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:06.972620 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06736ab1-20ca-4a48-afb5-c2f50139da2e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:07.037566 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 340/686] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:05:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 09:05:07.061901 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30dafbc-bf04-4bc4-b4ef-ecb4101fa8de None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:07.062806 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06736ab1-20ca-4a48-afb5-c2f50139da2e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:07.064798 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06736ab1-20ca-4a48-afb5-c2f50139da2e None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:07.103832 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:07.204088 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 347/687] 10.222.0.26 () {58 vars in 1282 bytes} [Thu Jul 14 09:05:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 09:05:07.209763 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30dafbc-bf04-4bc4-b4ef-ecb4101fa8de None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:07.211946 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06736ab1-20ca-4a48-afb5-c2f50139da2e None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:07.254362 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:07.279142 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3c44eb-a659-4140-a4ff-b66f82b760b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:07.280036 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3c44eb-a659-4140-a4ff-b66f82b760b2 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:07.280314 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3c44eb-a659-4140-a4ff-b66f82b760b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:07.280629 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3c44eb-a659-4140-a4ff-b66f82b760b2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:07.302451 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3c44eb-a659-4140-a4ff-b66f82b760b2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 09:05:07.315331 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-cb3c44eb-a659-4140-a4ff-b66f82b760b2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 09:05:07.315331 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:05:07.315331 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:05:07.315331 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:05:07.315331 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:05:07.315331 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:05:07.315331 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:05:07.315331 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:05:07.315331 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:05:07.315331 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:05:07.315331 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:05:07.315331 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:05:07.315331 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 09:05:07.315331 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 09:05:07.315331 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 09:05:07.315331 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 09:05:07.316313 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 09:05:07.316313 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 09:05:07.316313 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 09:05:07.316313 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 09:05:07.316313 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:05:07.316313 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:05:07.316313 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:05:07.316313 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:05:07.316313 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:05:07.316313 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:05:07.316313 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:05:07.316313 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 14 09:05:07.316313 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:05:07.316313 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:05:07.316313 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:05:07.316313 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:05:07.317171 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:05:07.317171 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:05:07.317171 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:05:07.317171 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:05:07.317171 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 09:05:07.317171 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 09:05:07.317171 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 09:05:07.317171 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 09:05:07.317171 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 09:05:07.317171 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 09:05:07.317171 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 09:05:07.317171 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 14 09:05:07.317833 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 341/688] 10.222.0.26 () {56 vars in 1096 bytes} [Thu Jul 14 09:05:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:05:07.324027 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3c44eb-a659-4140-a4ff-b66f82b760b2 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:07.401407 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06736ab1-20ca-4a48-afb5-c2f50139da2e None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:07.439418 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c144da-4d0d-4a5d-b0ff-a528fdd07f26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:07.440222 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c144da-4d0d-4a5d-b0ff-a528fdd07f26 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:07.440502 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c144da-4d0d-4a5d-b0ff-a528fdd07f26 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:07.440774 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c144da-4d0d-4a5d-b0ff-a528fdd07f26 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:07.463088 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c144da-4d0d-4a5d-b0ff-a528fdd07f26 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:07.464580 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c144da-4d0d-4a5d-b0ff-a528fdd07f26 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:07.485470 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:07.498008 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-37c144da-4d0d-4a5d-b0ff-a528fdd07f26 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 09:05:07.498008 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:05:07.498008 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:05:07.498008 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:05:07.498008 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:05:07.498008 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:05:07.498008 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:05:07.498008 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:05:07.498008 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:05:07.498008 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:05:07.498008 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:05:07.498008 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:05:07.498008 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 09:05:07.498008 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 09:05:07.498008 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 09:05:07.498902 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 09:05:07.498902 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:05:07.498902 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:05:07.498902 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:05:07.498902 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:05:07.498902 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:05:07.498902 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:05:07.498902 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:05:07.498902 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 14 09:05:07.498902 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:05:07.498902 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:05:07.498902 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:05:07.498902 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:05:07.498902 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:05:07.498902 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:05:07.498902 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:05:07.498902 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:05:07.499836 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 09:05:07.499836 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 09:05:07.499836 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 09:05:07.499836 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 14 09:05:07.501999 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 348/689] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jul 14 09:05:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:05:07.506152 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c144da-4d0d-4a5d-b0ff-a528fdd07f26 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:07.530382 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3c44eb-a659-4140-a4ff-b66f82b760b2 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:07.598985 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a00d2e-3994-4257-885d-9400fe4a880b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:07.600069 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a00d2e-3994-4257-885d-9400fe4a880b None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:07.600615 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a00d2e-3994-4257-885d-9400fe4a880b None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:07.601095 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a00d2e-3994-4257-885d-9400fe4a880b None admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:07.602657 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a00d2e-3994-4257-885d-9400fe4a880b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:05:07.604240 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a00d2e-3994-4257-885d-9400fe4a880b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:07.607086 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a00d2e-3994-4257-885d-9400fe4a880b None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:07.625654 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 342/690] 10.222.0.26 () {56 vars in 1099 bytes} [Thu Jul 14 09:05:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:05:07.630948 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a00d2e-3994-4257-885d-9400fe4a880b None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:07.649547 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:07.680485 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c144da-4d0d-4a5d-b0ff-a528fdd07f26 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:07.726000 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2f653b-7b47-49b7-b778-95b41c9ec9f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:07.727072 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2f653b-7b47-49b7-b778-95b41c9ec9f4 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:07.727643 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2f653b-7b47-49b7-b778-95b41c9ec9f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:07.728190 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2f653b-7b47-49b7-b778-95b41c9ec9f4 None admin] PATH_INFO: `/v3/domains` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:07.729152 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2f653b-7b47-49b7-b778-95b41c9ec9f4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:05:07.729650 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2f653b-7b47-49b7-b778-95b41c9ec9f4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:07.731245 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2f653b-7b47-49b7-b778-95b41c9ec9f4 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:07.754671 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 349/691] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Jul 14 09:05:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:05:07.768841 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2f653b-7b47-49b7-b778-95b41c9ec9f4 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:07.775553 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a00d2e-3994-4257-885d-9400fe4a880b None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:07.831025 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3028b529-e0f4-4b2b-ae45-35a47dc69397 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:07.831873 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3028b529-e0f4-4b2b-ae45-35a47dc69397 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:07.832159 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3028b529-e0f4-4b2b-ae45-35a47dc69397 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:07.832439 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3028b529-e0f4-4b2b-ae45-35a47dc69397 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:07.843985 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3028b529-e0f4-4b2b-ae45-35a47dc69397 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:07.852937 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3028b529-e0f4-4b2b-ae45-35a47dc69397 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:07.861730 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:07.880576 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-3028b529-e0f4-4b2b-ae45-35a47dc69397 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 09:05:07.880576 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:05:07.880576 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:05:07.880576 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:05:07.880576 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:05:07.880576 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:05:07.880576 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:05:07.880576 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:05:07.880576 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:05:07.880576 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:05:07.880576 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:05:07.880576 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:05:07.880576 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 09:05:07.880576 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 09:05:07.880576 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 09:05:07.881618 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 09:05:07.881618 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:05:07.881618 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:05:07.881618 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:05:07.881618 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:05:07.881618 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:05:07.881618 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:05:07.881618 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:05:07.881618 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 14 09:05:07.881618 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:05:07.881618 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:05:07.881618 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:05:07.881618 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:05:07.881618 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:05:07.881618 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:05:07.881618 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:05:07.881618 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:05:07.882522 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 09:05:07.882522 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 09:05:07.882522 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 09:05:07.882522 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 14 09:05:07.882522 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 343/692] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jul 14 09:05:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 09:05:07.896196 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3028b529-e0f4-4b2b-ae45-35a47dc69397 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:07.904685 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2f653b-7b47-49b7-b778-95b41c9ec9f4 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:07.949103 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cfbe87-25e9-4654-9243-a792fb5bfb08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:07.950786 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cfbe87-25e9-4654-9243-a792fb5bfb08 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:07.951087 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cfbe87-25e9-4654-9243-a792fb5bfb08 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:07.951390 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cfbe87-25e9-4654-9243-a792fb5bfb08 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:07.960201 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cfbe87-25e9-4654-9243-a792fb5bfb08 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:07.961720 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cfbe87-25e9-4654-9243-a792fb5bfb08 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:07.963763 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:07.984499 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-49cfbe87-25e9-4654-9243-a792fb5bfb08 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 09:05:07.984499 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:05:07.984499 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:05:07.984499 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:05:07.984499 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:05:07.984499 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:05:07.984499 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:05:07.984499 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:05:07.984499 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:05:07.984499 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:05:07.984499 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:05:07.984499 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:05:07.984499 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 09:05:07.984499 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 09:05:07.984499 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 09:05:07.985416 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 09:05:07.985416 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:05:07.985416 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:05:07.985416 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:05:07.985416 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:05:07.985416 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:05:07.985416 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:05:07.985416 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:05:07.985416 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 14 09:05:07.985416 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:05:07.985416 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:05:07.985416 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:05:07.985416 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:05:07.985416 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:05:07.985416 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:05:07.985416 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:05:07.985416 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:05:07.986314 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 09:05:07.986314 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 09:05:07.986314 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 09:05:07.986314 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 09:05:07.986314 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 09:05:07.986314 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 09:05:07.986314 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 09:05:07.986314 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 14 09:05:07.986314 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 350/693] 10.222.0.26 () {56 vars in 1111 bytes} [Thu Jul 14 09:05:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:05:07.992147 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cfbe87-25e9-4654-9243-a792fb5bfb08 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:08.123865 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3028b529-e0f4-4b2b-ae45-35a47dc69397 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:08.177147 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cfbe87-25e9-4654-9243-a792fb5bfb08 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:08.188122 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ea5506-8ac1-4216-9f13-35f4f1922edf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:08.188796 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ea5506-8ac1-4216-9f13-35f4f1922edf None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:08.189079 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ea5506-8ac1-4216-9f13-35f4f1922edf None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:08.189351 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ea5506-8ac1-4216-9f13-35f4f1922edf None admin] PATH_INFO: `/v3/domains` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:08.190082 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ea5506-8ac1-4216-9f13-35f4f1922edf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:05:08.190389 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ea5506-8ac1-4216-9f13-35f4f1922edf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:08.191800 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ea5506-8ac1-4216-9f13-35f4f1922edf None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:08.212380 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 344/694] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Jul 14 09:05:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:05:08.218420 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ea5506-8ac1-4216-9f13-35f4f1922edf None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:08.250264 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81dc2eec-5fcb-4a1e-9ada-5dbf7504293a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:08.250954 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dc2eec-5fcb-4a1e-9ada-5dbf7504293a None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:08.252786 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dc2eec-5fcb-4a1e-9ada-5dbf7504293a None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:08.253189 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dc2eec-5fcb-4a1e-9ada-5dbf7504293a None admin] PATH_INFO: `/v3/projects` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:08.254064 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81dc2eec-5fcb-4a1e-9ada-5dbf7504293a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:05:08.254382 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81dc2eec-5fcb-4a1e-9ada-5dbf7504293a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:08.255978 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81dc2eec-5fcb-4a1e-9ada-5dbf7504293a None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:08.287524 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 351/695] 10.222.0.26 () {56 vars in 1148 bytes} [Thu Jul 14 09:05:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:05:08.289402 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fb749f-cc63-48c1-92e6-47c6844e64b6 None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:08.289724 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fb749f-cc63-48c1-92e6-47c6844e64b6 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:08.290003 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fb749f-cc63-48c1-92e6-47c6844e64b6 None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:08.291540 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 352/696] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:05:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:05:08.297318 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9c71d9-f2a7-4f10-9cee-2e895d4f78ea None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:08.297652 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9c71d9-f2a7-4f10-9cee-2e895d4f78ea None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:08.297935 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9c71d9-f2a7-4f10-9cee-2e895d4f78ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:08.318562 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-bc9c71d9-f2a7-4f10-9cee-2e895d4f78ea None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:05:08.319015 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:08.320120 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-bc9c71d9-f2a7-4f10-9cee-2e895d4f78ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:05:08.363851 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ea5506-8ac1-4216-9f13-35f4f1922edf None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:08.383768 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:08.399670 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c799bd68-2a41-4dd3-a6e5-62e87d7ffc38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:08.400521 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c799bd68-2a41-4dd3-a6e5-62e87d7ffc38 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:08.400848 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c799bd68-2a41-4dd3-a6e5-62e87d7ffc38 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:08.401126 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c799bd68-2a41-4dd3-a6e5-62e87d7ffc38 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:08.411587 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 353/697] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:05:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 09:05:08.412620 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9c71d9-f2a7-4f10-9cee-2e895d4f78ea None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:08.414057 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c799bd68-2a41-4dd3-a6e5-62e87d7ffc38 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:08.417360 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c799bd68-2a41-4dd3-a6e5-62e87d7ffc38 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:08.436205 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-c799bd68-2a41-4dd3-a6e5-62e87d7ffc38 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 14 09:05:08.436205 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:05:08.436205 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:05:08.436205 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:05:08.436205 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:05:08.436205 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:05:08.436205 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:05:08.436205 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:05:08.436205 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:05:08.436205 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:05:08.436205 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:05:08.436205 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:05:08.436205 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 09:05:08.436205 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 09:05:08.436205 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 09:05:08.436205 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 09:05:08.437241 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:05:08.437241 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:05:08.437241 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 09:05:08.437241 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 09:05:08.437241 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 09:05:08.437241 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 09:05:08.437241 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:05:08.437241 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:05:08.437241 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:05:08.437241 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:05:08.437241 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:05:08.437241 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 14 09:05:08.437241 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:05:08.437241 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:05:08.437241 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:05:08.437241 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:05:08.437241 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:05:08.437241 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:05:08.438254 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:05:08.438254 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:05:08.438254 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 09:05:08.438254 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 09:05:08.438254 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 09:05:08.438254 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 09:05:08.438254 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 09:05:08.438254 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 09:05:08.438254 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 14 09:05:08.438254 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 14 09:05:08.438254 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 345/698] 10.222.0.26 () {56 vars in 1093 bytes} [Thu Jul 14 09:05:08 2022] GET /identity/v3/users/nova => generated 82 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:05:08.438862 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c799bd68-2a41-4dd3-a6e5-62e87d7ffc38 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:08.545721 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c799bd68-2a41-4dd3-a6e5-62e87d7ffc38 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:08.557926 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9c71d9-f2a7-4f10-9cee-2e895d4f78ea None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:08.569051 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:08.589017 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0d420b-5052-4c07-98d7-7976c9459bc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:08.590240 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0d420b-5052-4c07-98d7-7976c9459bc6 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:08.590625 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0d420b-5052-4c07-98d7-7976c9459bc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:08.591000 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0d420b-5052-4c07-98d7-7976c9459bc6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:08.600610 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b38638f-a610-4f0a-9bfd-55b368c3cc14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:08.601505 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b38638f-a610-4f0a-9bfd-55b368c3cc14 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:08.601942 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b38638f-a610-4f0a-9bfd-55b368c3cc14 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:08.602455 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b38638f-a610-4f0a-9bfd-55b368c3cc14 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:08.603600 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b38638f-a610-4f0a-9bfd-55b368c3cc14 None admin] RBAC: Adding query filter params (role.id=f5c487bc605f45df82ef6367c80fae70, scope.project.id=33bf210d015847439655b9943d1b0261, user.id=e5a43c80514b4a13af03c7f8ebcf7a5f) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:05:08.604055 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b38638f-a610-4f0a-9bfd-55b368c3cc14 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:08.605243 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0d420b-5052-4c07-98d7-7976c9459bc6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:08.605896 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b38638f-a610-4f0a-9bfd-55b368c3cc14 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:08.606828 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0d420b-5052-4c07-98d7-7976c9459bc6 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:08.619450 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 354/699] 10.222.0.26 () {56 vars in 1374 bytes} [Thu Jul 14 09:05:08 2022] GET /identity/v3/role_assignments?user.id=e5a43c80514b4a13af03c7f8ebcf7a5f&scope.project.id=33bf210d015847439655b9943d1b0261&role.id=f5c487bc605f45df82ef6367c80fae70 => generated 608 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:05:08.620786 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b38638f-a610-4f0a-9bfd-55b368c3cc14 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:08.626680 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-8c0d420b-5052-4c07-98d7-7976c9459bc6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 09:05:08.626680 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:05:08.626680 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:05:08.626680 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:05:08.626680 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:05:08.626680 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:05:08.626680 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:05:08.626680 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:05:08.626680 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:05:08.626680 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:05:08.626680 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:05:08.626680 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:05:08.626680 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 09:05:08.626680 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 09:05:08.626680 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 09:05:08.627679 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 09:05:08.627679 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:05:08.627679 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:05:08.627679 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:05:08.627679 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:05:08.627679 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:05:08.627679 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:05:08.627679 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:05:08.627679 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 14 09:05:08.627679 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:05:08.627679 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:05:08.627679 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:05:08.627679 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:05:08.627679 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:05:08.627679 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:05:08.627679 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:05:08.627679 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:05:08.628897 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 09:05:08.628897 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 09:05:08.628897 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 09:05:08.628897 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 14 09:05:08.628897 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 346/700] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jul 14 09:05:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:05:08.635426 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0d420b-5052-4c07-98d7-7976c9459bc6 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:08.754621 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0d420b-5052-4c07-98d7-7976c9459bc6 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:08.772663 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:08.797638 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0252a7-8c00-4f7a-ad3f-5cd45a9e3c86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:08.798823 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0252a7-8c00-4f7a-ad3f-5cd45a9e3c86 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:08.799693 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0252a7-8c00-4f7a-ad3f-5cd45a9e3c86 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:08.800195 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0252a7-8c00-4f7a-ad3f-5cd45a9e3c86 None admin] PATH_INFO: `/v3/domains` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:08.801168 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0252a7-8c00-4f7a-ad3f-5cd45a9e3c86 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:05:08.801695 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0252a7-8c00-4f7a-ad3f-5cd45a9e3c86 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:08.803528 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0252a7-8c00-4f7a-ad3f-5cd45a9e3c86 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:08.818172 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 347/701] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Jul 14 09:05:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:05:08.824918 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0252a7-8c00-4f7a-ad3f-5cd45a9e3c86 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:08.863627 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:08.881238 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b38638f-a610-4f0a-9bfd-55b368c3cc14 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:08.925463 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff701b7-53c9-4f0a-b2ae-2294c300da15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:08.925463 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff701b7-53c9-4f0a-b2ae-2294c300da15 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:08.925730 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff701b7-53c9-4f0a-b2ae-2294c300da15 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:08.926232 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff701b7-53c9-4f0a-b2ae-2294c300da15 None admin] PATH_INFO: `/v3/users` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:08.926889 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff701b7-53c9-4f0a-b2ae-2294c300da15 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:05:08.927892 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff701b7-53c9-4f0a-b2ae-2294c300da15 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:08.928817 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff701b7-53c9-4f0a-b2ae-2294c300da15 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:08.950374 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 355/702] 10.222.0.26 () {56 vars in 1133 bytes} [Thu Jul 14 09:05:08 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:05:08.951709 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f07ef4-bf0c-404d-9eba-ac4bd6a6114b None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:08.952084 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f07ef4-bf0c-404d-9eba-ac4bd6a6114b None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:08.952381 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f07ef4-bf0c-404d-9eba-ac4bd6a6114b None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:08.952808 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0252a7-8c00-4f7a-ad3f-5cd45a9e3c86 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:08.953918 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 356/703] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:05:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:05:08.957334 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f07ef4-bf0c-404d-9eba-ac4bd6a6114b None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:08.995555 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1828be-b6f3-4781-91d9-c041f9980eb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:08.996229 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1828be-b6f3-4781-91d9-c041f9980eb9 None admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:08.996599 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1828be-b6f3-4781-91d9-c041f9980eb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:08.996882 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1828be-b6f3-4781-91d9-c041f9980eb9 None admin] PATH_INFO: `/v3/users` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:08.997601 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1828be-b6f3-4781-91d9-c041f9980eb9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:08.999139 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1828be-b6f3-4781-91d9-c041f9980eb9 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:09.023417 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 348/704] 10.222.0.26 () {60 vars in 1132 bytes} [Thu Jul 14 09:05:08 2022] POST /identity/v3/users => generated 254 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 09:05:09.027365 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00fe633-84d9-4bf4-9c2d-f5d0cae40773 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:09.027365 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00fe633-84d9-4bf4-9c2d-f5d0cae40773 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:09.027365 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00fe633-84d9-4bf4-9c2d-f5d0cae40773 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:09.067851 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-f00fe633-84d9-4bf4-9c2d-f5d0cae40773 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:05:09.071021 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-f00fe633-84d9-4bf4-9c2d-f5d0cae40773 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:05:09.130959 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f07ef4-bf0c-404d-9eba-ac4bd6a6114b None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:09.143592 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:09.174052 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f33a8b4-02ec-4ffd-b79e-40cb63806cae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:09.175021 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f33a8b4-02ec-4ffd-b79e-40cb63806cae None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:09.175755 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f33a8b4-02ec-4ffd-b79e-40cb63806cae None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:09.176473 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f33a8b4-02ec-4ffd-b79e-40cb63806cae None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:09.223084 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 349/705] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:05:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 09:05:09.227276 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb91a99d-d5eb-47a8-8798-d6b683a653c6 None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:09.227276 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb91a99d-d5eb-47a8-8798-d6b683a653c6 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:09.227276 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb91a99d-d5eb-47a8-8798-d6b683a653c6 None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:09.228026 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 350/706] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:05:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:05:09.233782 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cf5faa-4276-4c04-a534-b0012c8226dd None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:09.234064 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cf5faa-4276-4c04-a534-b0012c8226dd None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:09.234334 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cf5faa-4276-4c04-a534-b0012c8226dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:09.251143 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-49cf5faa-4276-4c04-a534-b0012c8226dd None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:05:09.252321 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-49cf5faa-4276-4c04-a534-b0012c8226dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:05:09.267711 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:09.268773 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f33a8b4-02ec-4ffd-b79e-40cb63806cae None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:09.269949 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f33a8b4-02ec-4ffd-b79e-40cb63806cae None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:09.349984 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 351/707] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:05:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 09:05:09.350656 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cf5faa-4276-4c04-a534-b0012c8226dd None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:09.384437 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 357/708] 10.222.0.26 () {58 vars in 1282 bytes} [Thu Jul 14 09:05:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 14 09:05:09.385344 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f33a8b4-02ec-4ffd-b79e-40cb63806cae None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:09.396425 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:09.471466 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cf5faa-4276-4c04-a534-b0012c8226dd None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:09.505052 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329481d0-e4c4-4bd8-af65-0bc6dc870bfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:09.505865 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329481d0-e4c4-4bd8-af65-0bc6dc870bfd None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:09.506181 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329481d0-e4c4-4bd8-af65-0bc6dc870bfd None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:09.506454 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329481d0-e4c4-4bd8-af65-0bc6dc870bfd None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:09.507107 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329481d0-e4c4-4bd8-af65-0bc6dc870bfd None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:09.507657 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f33a8b4-02ec-4ffd-b79e-40cb63806cae None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:09.508540 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329481d0-e4c4-4bd8-af65-0bc6dc870bfd None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:09.522547 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-329481d0-e4c4-4bd8-af65-0bc6dc870bfd None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 14 09:05:09.522547 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:05:09.522547 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 09:05:09.522547 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 09:05:09.522547 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 09:05:09.522547 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 09:05:09.522547 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 09:05:09.522547 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 09:05:09.522547 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 14 09:05:09.522547 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 14 09:05:09.522547 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 09:05:09.522547 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 14 09:05:09.522547 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:05:09.522547 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:05:09.522547 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:05:09.522547 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:05:09.524055 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:05:09.524055 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:05:09.524055 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:05:09.524055 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:05:09.524055 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:05:09.524055 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:05:09.524055 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:05:09.524055 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 09:05:09.524055 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 09:05:09.524055 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 09:05:09.524055 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 09:05:09.524055 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:05:09.524055 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:05:09.524055 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:05:09.524055 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:05:09.524055 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:05:09.524932 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:05:09.524932 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:05:09.524932 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 14 09:05:09.524932 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:05:09.524932 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:05:09.524932 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:05:09.524932 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:05:09.524932 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:05:09.524932 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:05:09.524932 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:05:09.524932 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:05:09.524932 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 09:05:09.524932 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 09:05:09.524932 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 14 09:05:09.524932 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 14 09:05:09.528726 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 352/709] 10.222.0.26 () {56 vars in 1129 bytes} [Thu Jul 14 09:05:09 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:05:09.532577 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329481d0-e4c4-4bd8-af65-0bc6dc870bfd None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:09.555420 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94c8bbd-85f7-47f5-a0b4-b4f32ec880ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:09.556716 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94c8bbd-85f7-47f5-a0b4-b4f32ec880ad None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:09.557321 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94c8bbd-85f7-47f5-a0b4-b4f32ec880ad None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:09.557792 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94c8bbd-85f7-47f5-a0b4-b4f32ec880ad None admin] PATH_INFO: `/v3/services/network` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:09.558927 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94c8bbd-85f7-47f5-a0b4-b4f32ec880ad None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:09.560940 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94c8bbd-85f7-47f5-a0b4-b4f32ec880ad None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:09.567601 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:09.581931 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-a94c8bbd-85f7-47f5-a0b4-b4f32ec880ad None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 14 09:05:09.581931 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:05:09.581931 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 09:05:09.581931 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 09:05:09.581931 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 09:05:09.581931 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 09:05:09.581931 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 09:05:09.581931 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 09:05:09.581931 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 14 09:05:09.581931 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 14 09:05:09.581931 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 09:05:09.581931 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 14 09:05:09.581931 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:05:09.581931 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:05:09.581931 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:05:09.581931 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:05:09.581931 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:05:09.583028 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:05:09.583028 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:05:09.583028 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:05:09.583028 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:05:09.583028 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:05:09.583028 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:05:09.583028 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 09:05:09.583028 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 09:05:09.583028 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 09:05:09.583028 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 09:05:09.583028 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:05:09.583028 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:05:09.583028 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:05:09.583028 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:05:09.583028 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:05:09.583028 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:05:09.584333 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:05:09.584333 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 14 09:05:09.584333 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:05:09.584333 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:05:09.584333 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:05:09.584333 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:05:09.584333 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:05:09.584333 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:05:09.584333 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:05:09.584333 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:05:09.584333 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 09:05:09.584333 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 09:05:09.584333 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 14 09:05:09.584333 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 14 09:05:09.584333 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 358/710] 10.222.0.26 () {56 vars in 1111 bytes} [Thu Jul 14 09:05:09 2022] GET /identity/v3/services/network => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:05:09.585160 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94c8bbd-85f7-47f5-a0b4-b4f32ec880ad None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:09.661419 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:09.666613 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329481d0-e4c4-4bd8-af65-0bc6dc870bfd None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:09.700954 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94c8bbd-85f7-47f5-a0b4-b4f32ec880ad None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:09.702737 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c56cead-2b3f-4f7d-9f75-ad0b4ace26d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:09.703904 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c56cead-2b3f-4f7d-9f75-ad0b4ace26d3 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:09.704835 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c56cead-2b3f-4f7d-9f75-ad0b4ace26d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:09.704835 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c56cead-2b3f-4f7d-9f75-ad0b4ace26d3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:09.716005 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c56cead-2b3f-4f7d-9f75-ad0b4ace26d3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:09.717553 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c56cead-2b3f-4f7d-9f75-ad0b4ace26d3 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:09.737797 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-9c56cead-2b3f-4f7d-9f75-ad0b4ace26d3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 09:05:09.737797 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:05:09.737797 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:05:09.737797 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:05:09.737797 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:05:09.737797 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:05:09.737797 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:05:09.737797 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:05:09.737797 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:05:09.737797 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:05:09.737797 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:05:09.737797 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:05:09.737797 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 09:05:09.737797 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 09:05:09.737797 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 09:05:09.738746 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 09:05:09.738746 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:05:09.738746 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:05:09.738746 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:05:09.738746 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:05:09.738746 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:05:09.738746 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:05:09.738746 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:05:09.738746 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 14 09:05:09.738746 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:05:09.738746 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:05:09.738746 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:05:09.738746 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:05:09.738746 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:05:09.738746 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:05:09.738746 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:05:09.738746 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:05:09.739874 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 09:05:09.739874 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 09:05:09.739874 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 09:05:09.739874 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 14 09:05:09.739874 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 353/711] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jul 14 09:05:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:05:09.740866 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c56cead-2b3f-4f7d-9f75-ad0b4ace26d3 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:09.744435 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e4ff52-f40e-4d5f-86ef-4ab661360b91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:09.746310 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e4ff52-f40e-4d5f-86ef-4ab661360b91 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:09.746783 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e4ff52-f40e-4d5f-86ef-4ab661360b91 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:09.747260 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e4ff52-f40e-4d5f-86ef-4ab661360b91 None admin] PATH_INFO: `/v3/services` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:09.748163 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e4ff52-f40e-4d5f-86ef-4ab661360b91 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:05:09.748647 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e4ff52-f40e-4d5f-86ef-4ab661360b91 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:09.750242 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e4ff52-f40e-4d5f-86ef-4ab661360b91 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:09.757421 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 359/712] 10.222.0.26 () {56 vars in 1124 bytes} [Thu Jul 14 09:05:09 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:05:09.758451 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e4ff52-f40e-4d5f-86ef-4ab661360b91 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:09.830330 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:09.878196 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c56cead-2b3f-4f7d-9f75-ad0b4ace26d3 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:09.893988 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e4ff52-f40e-4d5f-86ef-4ab661360b91 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:09.919530 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719f1576-2a98-4e4c-ab87-e0db06f55c0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:09.920786 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719f1576-2a98-4e4c-ab87-e0db06f55c0d None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:09.921160 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719f1576-2a98-4e4c-ab87-e0db06f55c0d None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:09.921522 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719f1576-2a98-4e4c-ab87-e0db06f55c0d None admin] PATH_INFO: `/v3/services` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:09.922321 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719f1576-2a98-4e4c-ab87-e0db06f55c0d None admin] RBAC: Adding query filter params (name=network) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:05:09.922738 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719f1576-2a98-4e4c-ab87-e0db06f55c0d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:09.924238 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719f1576-2a98-4e4c-ab87-e0db06f55c0d None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:09.929299 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 354/713] 10.222.0.26 () {56 vars in 1112 bytes} [Thu Jul 14 09:05:09 2022] GET /identity/v3/services?name=network => generated 124 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:05:09.930463 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719f1576-2a98-4e4c-ab87-e0db06f55c0d None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:09.943993 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6e1584-ec0f-4347-b8e1-032caecaac99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:09.945011 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6e1584-ec0f-4347-b8e1-032caecaac99 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:09.945475 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6e1584-ec0f-4347-b8e1-032caecaac99 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:09.946164 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6e1584-ec0f-4347-b8e1-032caecaac99 None admin] PATH_INFO: `/v3/domains` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:09.947598 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6e1584-ec0f-4347-b8e1-032caecaac99 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:05:09.948066 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6e1584-ec0f-4347-b8e1-032caecaac99 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:09.950148 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6e1584-ec0f-4347-b8e1-032caecaac99 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:09.953297 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:09.969930 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 360/714] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Jul 14 09:05:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:05:09.971022 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6e1584-ec0f-4347-b8e1-032caecaac99 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:10.043366 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719f1576-2a98-4e4c-ab87-e0db06f55c0d None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:10.070806 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:10.072999 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6e1584-ec0f-4347-b8e1-032caecaac99 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:10.080528 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8faadb-9b85-4754-829e-ec0d4b1d4227 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:10.081323 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8faadb-9b85-4754-829e-ec0d4b1d4227 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:10.082060 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8faadb-9b85-4754-829e-ec0d4b1d4227 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:10.082060 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8faadb-9b85-4754-829e-ec0d4b1d4227 None admin] PATH_INFO: `/v3/services` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:10.082929 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8faadb-9b85-4754-829e-ec0d4b1d4227 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:05:10.083368 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8faadb-9b85-4754-829e-ec0d4b1d4227 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:10.084974 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8faadb-9b85-4754-829e-ec0d4b1d4227 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:10.099600 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 355/715] 10.222.0.26 () {56 vars in 1124 bytes} [Thu Jul 14 09:05:09 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:05:10.100733 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8faadb-9b85-4754-829e-ec0d4b1d4227 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:10.124660 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487dd9e2-a9b4-4ef0-b04e-8a4e6c98e0b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:10.125602 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487dd9e2-a9b4-4ef0-b04e-8a4e6c98e0b8 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:10.126034 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487dd9e2-a9b4-4ef0-b04e-8a4e6c98e0b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:10.126461 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487dd9e2-a9b4-4ef0-b04e-8a4e6c98e0b8 None admin] PATH_INFO: `/v3/services` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:10.127367 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487dd9e2-a9b4-4ef0-b04e-8a4e6c98e0b8 None admin] RBAC: Adding query filter params (type=network) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:05:10.127836 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487dd9e2-a9b4-4ef0-b04e-8a4e6c98e0b8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:10.135217 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487dd9e2-a9b4-4ef0-b04e-8a4e6c98e0b8 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:10.147347 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 361/716] 10.222.0.26 () {56 vars in 1112 bytes} [Thu Jul 14 09:05:09 2022] GET /identity/v3/services?type=network => generated 124 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:05:10.223741 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8faadb-9b85-4754-829e-ec0d4b1d4227 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:10.266614 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57954e4-4fa3-4df2-9696-29de3ae06608 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:10.267462 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57954e4-4fa3-4df2-9696-29de3ae06608 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:10.267741 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57954e4-4fa3-4df2-9696-29de3ae06608 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:10.268014 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57954e4-4fa3-4df2-9696-29de3ae06608 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:10.279251 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57954e4-4fa3-4df2-9696-29de3ae06608 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:10.280770 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57954e4-4fa3-4df2-9696-29de3ae06608 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:10.311207 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-e57954e4-4fa3-4df2-9696-29de3ae06608 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 09:05:10.311207 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:05:10.311207 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:05:10.311207 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:05:10.311207 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:05:10.311207 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:05:10.311207 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:05:10.311207 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:05:10.311207 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:05:10.311207 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:05:10.311207 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:05:10.311207 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:05:10.311207 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 09:05:10.311207 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 09:05:10.311207 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 09:05:10.312147 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 09:05:10.312147 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:05:10.312147 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:05:10.312147 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:05:10.312147 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:05:10.312147 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:05:10.312147 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:05:10.312147 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:05:10.312147 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 14 09:05:10.312147 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:05:10.312147 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:05:10.312147 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:05:10.312147 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:05:10.312147 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:05:10.312147 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:05:10.312147 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:05:10.312147 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:05:10.313742 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 09:05:10.313742 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 09:05:10.313742 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 09:05:10.313742 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 09:05:10.313742 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 09:05:10.313742 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 09:05:10.313742 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 09:05:10.313742 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 14 09:05:10.313742 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 356/717] 10.222.0.26 () {56 vars in 1111 bytes} [Thu Jul 14 09:05:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:05:10.318371 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487dd9e2-a9b4-4ef0-b04e-8a4e6c98e0b8 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:10.397231 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:10.443336 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487dd9e2-a9b4-4ef0-b04e-8a4e6c98e0b8 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:10.477951 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9436dca4-6194-4236-8ca5-92020e45678b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:10.478617 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9436dca4-6194-4236-8ca5-92020e45678b None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:10.478895 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9436dca4-6194-4236-8ca5-92020e45678b None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:10.479167 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9436dca4-6194-4236-8ca5-92020e45678b None admin] PATH_INFO: `/v3/projects` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:10.480040 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9436dca4-6194-4236-8ca5-92020e45678b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:05:10.480348 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9436dca4-6194-4236-8ca5-92020e45678b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:10.481781 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9436dca4-6194-4236-8ca5-92020e45678b None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:10.494828 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 362/718] 10.222.0.26 () {56 vars in 1148 bytes} [Thu Jul 14 09:05:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:05:10.500323 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57954e4-4fa3-4df2-9696-29de3ae06608 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:10.556331 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:10.604653 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57954e4-4fa3-4df2-9696-29de3ae06608 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:10.638292 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d352c21-ef42-481c-bb35-67426a94ac4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:10.638953 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d352c21-ef42-481c-bb35-67426a94ac4e None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:10.639256 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d352c21-ef42-481c-bb35-67426a94ac4e None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:10.639534 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d352c21-ef42-481c-bb35-67426a94ac4e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:10.640369 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d352c21-ef42-481c-bb35-67426a94ac4e None admin] RBAC: Adding query filter params (role.id=7b549d58d1724395b343609d895c80d0, scope.project.id=33bf210d015847439655b9943d1b0261, user.id=538fff3c5a2b487eb44ae7c1a1198102) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:05:10.640678 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d352c21-ef42-481c-bb35-67426a94ac4e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:10.642094 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d352c21-ef42-481c-bb35-67426a94ac4e None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:10.648164 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 357/719] 10.222.0.26 () {56 vars in 1374 bytes} [Thu Jul 14 09:05:10 2022] GET /identity/v3/role_assignments?user.id=538fff3c5a2b487eb44ae7c1a1198102&scope.project.id=33bf210d015847439655b9943d1b0261&role.id=7b549d58d1724395b343609d895c80d0 => generated 259 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 09:05:11.020424 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79da32c-e276-46c1-84d6-98d8da1c447d None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:11.020992 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79da32c-e276-46c1-84d6-98d8da1c447d None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:11.020992 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79da32c-e276-46c1-84d6-98d8da1c447d None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:11.022605 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 363/720] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:05:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:05:11.027827 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41ed68e-e1d6-4731-a31a-6a749f242eed None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:11.028121 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41ed68e-e1d6-4731-a31a-6a749f242eed None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:11.028402 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41ed68e-e1d6-4731-a31a-6a749f242eed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:11.046863 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-d41ed68e-e1d6-4731-a31a-6a749f242eed None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:05:11.048100 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-d41ed68e-e1d6-4731-a31a-6a749f242eed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:05:11.133478 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 358/721] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:05:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 09:05:11.168594 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79da32c-e276-46c1-84d6-98d8da1c447d None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:11.222917 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f67303-2a9d-47c7-b274-d02a169b8032 None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:11.223294 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f67303-2a9d-47c7-b274-d02a169b8032 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:11.223512 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f67303-2a9d-47c7-b274-d02a169b8032 None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:11.225135 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 359/722] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:05:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:05:11.230475 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893f942a-b9e7-49cb-8242-05e620e6efbf None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:11.230755 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893f942a-b9e7-49cb-8242-05e620e6efbf None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:11.231027 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893f942a-b9e7-49cb-8242-05e620e6efbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:11.256950 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-893f942a-b9e7-49cb-8242-05e620e6efbf None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:05:11.258150 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-893f942a-b9e7-49cb-8242-05e620e6efbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:05:11.334705 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:11.337942 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79da32c-e276-46c1-84d6-98d8da1c447d None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:11.371515 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:11.381335 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 360/723] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:05:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 09:05:11.400726 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe603dc-df95-415d-9148-d7b0cf958fe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:11.401543 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe603dc-df95-415d-9148-d7b0cf958fe3 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:11.401821 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe603dc-df95-415d-9148-d7b0cf958fe3 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:11.402089 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe603dc-df95-415d-9148-d7b0cf958fe3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:11.418818 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe603dc-df95-415d-9148-d7b0cf958fe3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 09:05:11.426495 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-8fe603dc-df95-415d-9148-d7b0cf958fe3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 09:05:11.426495 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:05:11.426495 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:05:11.426495 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:05:11.426495 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:05:11.426495 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:05:11.426495 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:05:11.426495 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:05:11.426495 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:05:11.426495 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:05:11.426495 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:05:11.426495 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:05:11.426495 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 09:05:11.426495 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 09:05:11.426495 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 09:05:11.426495 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 09:05:11.427517 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 09:05:11.427517 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 09:05:11.427517 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 09:05:11.427517 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 09:05:11.427517 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:05:11.427517 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:05:11.427517 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:05:11.427517 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:05:11.427517 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:05:11.427517 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:05:11.427517 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:05:11.427517 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 14 09:05:11.427517 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:05:11.427517 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:05:11.427517 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:05:11.427517 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:05:11.429010 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:05:11.429010 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:05:11.429010 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:05:11.429010 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:05:11.429010 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 09:05:11.429010 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 09:05:11.429010 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 09:05:11.429010 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 09:05:11.429010 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 09:05:11.429010 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 09:05:11.429010 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 09:05:11.429010 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 14 09:05:11.429010 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 364/724] 10.222.0.26 () {56 vars in 1096 bytes} [Thu Jul 14 09:05:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:05:11.433233 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893f942a-b9e7-49cb-8242-05e620e6efbf None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:11.436331 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe603dc-df95-415d-9148-d7b0cf958fe3 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:11.595588 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893f942a-b9e7-49cb-8242-05e620e6efbf None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:11.651515 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe603dc-df95-415d-9148-d7b0cf958fe3 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:11.671004 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab9e909-2b83-44dd-9dc5-a8453e0202d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:11.671895 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab9e909-2b83-44dd-9dc5-a8453e0202d0 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:11.672184 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab9e909-2b83-44dd-9dc5-a8453e0202d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:11.672455 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab9e909-2b83-44dd-9dc5-a8453e0202d0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:11.695034 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab9e909-2b83-44dd-9dc5-a8453e0202d0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 09:05:11.704984 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-2ab9e909-2b83-44dd-9dc5-a8453e0202d0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 09:05:11.704984 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:05:11.704984 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:05:11.704984 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:05:11.704984 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:05:11.704984 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:05:11.704984 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:05:11.704984 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:05:11.704984 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:05:11.704984 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:05:11.704984 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:05:11.704984 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:05:11.704984 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 09:05:11.704984 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 09:05:11.704984 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 09:05:11.704984 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 09:05:11.705945 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 09:05:11.705945 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 09:05:11.705945 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 09:05:11.705945 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 09:05:11.705945 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:05:11.705945 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:05:11.705945 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:05:11.705945 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:05:11.705945 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:05:11.705945 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:05:11.705945 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:05:11.705945 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 14 09:05:11.705945 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:05:11.705945 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:05:11.705945 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:05:11.705945 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:05:11.706910 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:05:11.706910 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:05:11.706910 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:05:11.706910 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:05:11.706910 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 09:05:11.706910 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 09:05:11.706910 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 09:05:11.706910 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 09:05:11.706910 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 09:05:11.706910 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 09:05:11.706910 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 09:05:11.706910 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 14 09:05:11.715375 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 361/725] 10.222.0.26 () {56 vars in 1102 bytes} [Thu Jul 14 09:05:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:05:11.715512 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9887d96e-0db2-4143-a8d7-3692b340e37d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:11.715608 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab9e909-2b83-44dd-9dc5-a8453e0202d0 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:11.716583 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9887d96e-0db2-4143-a8d7-3692b340e37d None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:11.716864 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9887d96e-0db2-4143-a8d7-3692b340e37d None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:11.717138 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9887d96e-0db2-4143-a8d7-3692b340e37d None admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:11.721528 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9887d96e-0db2-4143-a8d7-3692b340e37d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:05:11.721848 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9887d96e-0db2-4143-a8d7-3692b340e37d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:11.723280 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9887d96e-0db2-4143-a8d7-3692b340e37d None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:11.744168 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 365/726] 10.222.0.26 () {56 vars in 1099 bytes} [Thu Jul 14 09:05:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:05:11.747389 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9887d96e-0db2-4143-a8d7-3692b340e37d None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:11.796847 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:11.836987 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:11.875876 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab9e909-2b83-44dd-9dc5-a8453e0202d0 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:11.911620 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a753625-2cff-454c-9174-513fe335fad3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:11.912911 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a753625-2cff-454c-9174-513fe335fad3 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:11.913598 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a753625-2cff-454c-9174-513fe335fad3 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:11.914105 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a753625-2cff-454c-9174-513fe335fad3 None admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:11.915524 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a753625-2cff-454c-9174-513fe335fad3 None admin] RBAC: Adding query filter params (name=service) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:05:11.916025 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a753625-2cff-454c-9174-513fe335fad3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:11.917764 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a753625-2cff-454c-9174-513fe335fad3 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:11.926940 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 362/727] 10.222.0.26 () {56 vars in 1103 bytes} [Thu Jul 14 09:05:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:05:11.933172 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a753625-2cff-454c-9174-513fe335fad3 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:11.990353 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9887d96e-0db2-4143-a8d7-3692b340e37d None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:12.035997 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c48857-53f5-4d3c-b50c-b959137e9e9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:12.036810 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c48857-53f5-4d3c-b50c-b959137e9e9b None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:12.037070 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c48857-53f5-4d3c-b50c-b959137e9e9b None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:12.037340 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c48857-53f5-4d3c-b50c-b959137e9e9b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:12.049487 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c48857-53f5-4d3c-b50c-b959137e9e9b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:12.050941 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c48857-53f5-4d3c-b50c-b959137e9e9b None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:12.068054 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-31c48857-53f5-4d3c-b50c-b959137e9e9b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 09:05:12.068054 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:05:12.068054 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:05:12.068054 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:05:12.068054 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:05:12.068054 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:05:12.068054 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:05:12.068054 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:05:12.068054 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:05:12.068054 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:05:12.068054 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:05:12.068054 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:05:12.068054 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 09:05:12.068054 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 09:05:12.068054 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 09:05:12.068975 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 09:05:12.068975 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:05:12.068975 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:05:12.068975 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:05:12.068975 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:05:12.068975 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:05:12.068975 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:05:12.068975 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:05:12.068975 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 14 09:05:12.068975 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:05:12.068975 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:05:12.068975 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:05:12.068975 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:05:12.068975 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:05:12.068975 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:05:12.068975 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:05:12.068975 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:05:12.070418 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 09:05:12.070418 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 09:05:12.070418 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 09:05:12.070418 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 14 09:05:12.070418 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 366/728] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jul 14 09:05:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 322 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:05:12.074610 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c48857-53f5-4d3c-b50c-b959137e9e9b None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:12.135630 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:12.143847 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a753625-2cff-454c-9174-513fe335fad3 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:12.207148 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651bd6b7-a756-4c49-8f61-abc84def632c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:12.207986 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651bd6b7-a756-4c49-8f61-abc84def632c None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:12.208266 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651bd6b7-a756-4c49-8f61-abc84def632c None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:12.208537 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651bd6b7-a756-4c49-8f61-abc84def632c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:12.224104 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651bd6b7-a756-4c49-8f61-abc84def632c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:12.225567 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651bd6b7-a756-4c49-8f61-abc84def632c None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:12.252845 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-651bd6b7-a756-4c49-8f61-abc84def632c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 09:05:12.252845 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:05:12.252845 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:05:12.252845 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:05:12.252845 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:05:12.252845 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:05:12.252845 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:05:12.252845 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:05:12.252845 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:05:12.252845 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:05:12.252845 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:05:12.252845 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:05:12.252845 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 09:05:12.252845 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 09:05:12.252845 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 09:05:12.253948 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 09:05:12.253948 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:05:12.253948 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:05:12.253948 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:05:12.253948 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:05:12.253948 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:05:12.253948 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:05:12.253948 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:05:12.253948 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 14 09:05:12.253948 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:05:12.253948 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:05:12.253948 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:05:12.253948 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:05:12.253948 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:05:12.253948 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:05:12.253948 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:05:12.253948 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:05:12.255047 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 09:05:12.255047 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 09:05:12.255047 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 09:05:12.255047 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 14 09:05:12.255047 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 363/729] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jul 14 09:05:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 321 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 09:05:12.259077 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:12.265223 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c48857-53f5-4d3c-b50c-b959137e9e9b None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:12.267550 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651bd6b7-a756-4c49-8f61-abc84def632c None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:12.326776 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cdaabf-e1ce-47cc-9305-cb7fec8b4c91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:12.327813 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cdaabf-e1ce-47cc-9305-cb7fec8b4c91 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:12.328283 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cdaabf-e1ce-47cc-9305-cb7fec8b4c91 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:12.328725 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cdaabf-e1ce-47cc-9305-cb7fec8b4c91 None admin] PATH_INFO: `/v3/domains` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:12.329621 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cdaabf-e1ce-47cc-9305-cb7fec8b4c91 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:05:12.331619 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cdaabf-e1ce-47cc-9305-cb7fec8b4c91 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:12.333027 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cdaabf-e1ce-47cc-9305-cb7fec8b4c91 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:12.356872 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 367/730] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Jul 14 09:05:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:05:12.362360 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cdaabf-e1ce-47cc-9305-cb7fec8b4c91 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:12.402634 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651bd6b7-a756-4c49-8f61-abc84def632c None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:12.436608 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9e282c-1724-4a87-9dbc-1c8f20494401 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:12.437593 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9e282c-1724-4a87-9dbc-1c8f20494401 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:12.438047 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9e282c-1724-4a87-9dbc-1c8f20494401 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:12.438488 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9e282c-1724-4a87-9dbc-1c8f20494401 None admin] PATH_INFO: `/v3/domains` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:12.439406 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9e282c-1724-4a87-9dbc-1c8f20494401 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:05:12.439899 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9e282c-1724-4a87-9dbc-1c8f20494401 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:12.441421 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9e282c-1724-4a87-9dbc-1c8f20494401 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:12.453907 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 364/731] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Jul 14 09:05:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:05:12.458742 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9e282c-1724-4a87-9dbc-1c8f20494401 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:12.472652 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:12.538931 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cdaabf-e1ce-47cc-9305-cb7fec8b4c91 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:12.578345 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3741979-757e-457c-820c-ba0feeeb1022 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:12.578800 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9e282c-1724-4a87-9dbc-1c8f20494401 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:12.579678 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3741979-757e-457c-820c-ba0feeeb1022 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:12.580236 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3741979-757e-457c-820c-ba0feeeb1022 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:12.580722 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3741979-757e-457c-820c-ba0feeeb1022 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:12.588852 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3741979-757e-457c-820c-ba0feeeb1022 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:12.590714 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3741979-757e-457c-820c-ba0feeeb1022 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:12.606894 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-b3741979-757e-457c-820c-ba0feeeb1022 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 14 09:05:12.606894 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:05:12.606894 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:05:12.606894 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:05:12.606894 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:05:12.606894 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:05:12.606894 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:05:12.606894 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:05:12.606894 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:05:12.606894 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:05:12.606894 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:05:12.606894 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:05:12.606894 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 09:05:12.606894 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 09:05:12.606894 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 09:05:12.606894 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 09:05:12.607944 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:05:12.607944 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:05:12.607944 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 09:05:12.607944 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 09:05:12.607944 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 09:05:12.607944 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 09:05:12.607944 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:05:12.607944 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:05:12.607944 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:05:12.607944 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:05:12.607944 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:05:12.607944 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 14 09:05:12.607944 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:05:12.607944 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:05:12.607944 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:05:12.607944 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:05:12.607944 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:05:12.607944 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:05:12.608888 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:05:12.608888 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:05:12.608888 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 09:05:12.608888 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 09:05:12.608888 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 09:05:12.608888 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 09:05:12.608888 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 09:05:12.608888 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 09:05:12.608888 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 14 09:05:12.608888 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 14 09:05:12.615670 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 368/732] 10.222.0.26 () {56 vars in 1096 bytes} [Thu Jul 14 09:05:12 2022] GET /identity/v3/users/swift => generated 83 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 09:05:12.615815 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8047c5-b0a7-46b6-8c43-76e2c18d110c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:12.616477 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8047c5-b0a7-46b6-8c43-76e2c18d110c None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:12.616907 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8047c5-b0a7-46b6-8c43-76e2c18d110c None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:12.617244 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8047c5-b0a7-46b6-8c43-76e2c18d110c None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:12.618446 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed21f23c-05c0-4257-a042-ab6d039f2cd2 None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:12.618910 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed21f23c-05c0-4257-a042-ab6d039f2cd2 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:12.619490 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed21f23c-05c0-4257-a042-ab6d039f2cd2 None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:12.621568 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 369/733] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:05:12 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:05:12.622440 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed21f23c-05c0-4257-a042-ab6d039f2cd2 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:12.625310 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8047c5-b0a7-46b6-8c43-76e2c18d110c None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:12.627935 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8047c5-b0a7-46b6-8c43-76e2c18d110c None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:12.640530 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:12.653459 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-fd8047c5-b0a7-46b6-8c43-76e2c18d110c None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 14 09:05:12.653459 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:05:12.653459 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:05:12.653459 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:05:12.653459 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:05:12.653459 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:05:12.653459 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:05:12.653459 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:05:12.653459 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:05:12.653459 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:05:12.653459 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:05:12.653459 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:05:12.653459 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 09:05:12.653459 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 09:05:12.653459 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 09:05:12.653459 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 09:05:12.654484 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:05:12.654484 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:05:12.654484 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 09:05:12.654484 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 09:05:12.654484 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 09:05:12.654484 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 09:05:12.654484 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:05:12.654484 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:05:12.654484 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:05:12.654484 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:05:12.654484 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:05:12.654484 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 14 09:05:12.654484 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:05:12.654484 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:05:12.654484 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:05:12.654484 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:05:12.654484 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:05:12.654484 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:05:12.655467 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:05:12.655467 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:05:12.655467 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 09:05:12.655467 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 09:05:12.655467 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 09:05:12.655467 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 09:05:12.655467 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 09:05:12.655467 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 09:05:12.655467 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 14 09:05:12.655467 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 14 09:05:12.657739 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 365/734] 10.222.0.26 () {56 vars in 1117 bytes} [Thu Jul 14 09:05:12 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 09:05:12.658763 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11ac465-3465-4b7c-afc0-3419ad13a8c2 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:12.659045 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11ac465-3465-4b7c-afc0-3419ad13a8c2 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:12.659343 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11ac465-3465-4b7c-afc0-3419ad13a8c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:12.676927 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-c11ac465-3465-4b7c-afc0-3419ad13a8c2 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:05:12.678374 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-c11ac465-3465-4b7c-afc0-3419ad13a8c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:05:12.739299 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed21f23c-05c0-4257-a042-ab6d039f2cd2 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:12.744514 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:12.749752 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 366/735] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:05:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 09:05:12.750599 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11ac465-3465-4b7c-afc0-3419ad13a8c2 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:12.783151 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f93aed7-4c17-48eb-b89d-003a19d7b122 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:12.783975 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f93aed7-4c17-48eb-b89d-003a19d7b122 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:12.784262 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f93aed7-4c17-48eb-b89d-003a19d7b122 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:12.784704 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f93aed7-4c17-48eb-b89d-003a19d7b122 None admin] PATH_INFO: `/v3/users` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:12.785392 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f93aed7-4c17-48eb-b89d-003a19d7b122 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:05:12.785708 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f93aed7-4c17-48eb-b89d-003a19d7b122 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:12.787149 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f93aed7-4c17-48eb-b89d-003a19d7b122 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:12.807792 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 370/736] 10.222.0.26 () {56 vars in 1135 bytes} [Thu Jul 14 09:05:12 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:05:12.813379 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e3a0fa-2b13-4f83-8a0c-8b00c88c7c61 None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:12.813668 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e3a0fa-2b13-4f83-8a0c-8b00c88c7c61 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:12.813950 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e3a0fa-2b13-4f83-8a0c-8b00c88c7c61 None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:12.819238 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 371/737] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:05:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:05:12.819238 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e3a0fa-2b13-4f83-8a0c-8b00c88c7c61 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:12.862897 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11ac465-3465-4b7c-afc0-3419ad13a8c2 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:12.894996 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:12.902150 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5356dd-50d4-44aa-8af0-daf7fc570ebf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:12.903015 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5356dd-50d4-44aa-8af0-daf7fc570ebf None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:12.903326 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5356dd-50d4-44aa-8af0-daf7fc570ebf None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:12.903667 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5356dd-50d4-44aa-8af0-daf7fc570ebf None admin] PATH_INFO: `/v3/users` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:12.904449 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5356dd-50d4-44aa-8af0-daf7fc570ebf None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:05:12.904801 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5356dd-50d4-44aa-8af0-daf7fc570ebf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:12.906269 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5356dd-50d4-44aa-8af0-daf7fc570ebf None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:12.928998 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 367/738] 10.222.0.26 () {56 vars in 1149 bytes} [Thu Jul 14 09:05:12 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:05:12.928998 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5356dd-50d4-44aa-8af0-daf7fc570ebf None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:12.935641 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e3a0fa-2b13-4f83-8a0c-8b00c88c7c61 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:12.970945 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f901e7f6-27e0-45b0-9046-38902a7386a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:12.971651 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f901e7f6-27e0-45b0-9046-38902a7386a6 None admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:12.971941 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f901e7f6-27e0-45b0-9046-38902a7386a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:12.972219 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f901e7f6-27e0-45b0-9046-38902a7386a6 None admin] PATH_INFO: `/v3/services` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:12.972944 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f901e7f6-27e0-45b0-9046-38902a7386a6 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:12.974399 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f901e7f6-27e0-45b0-9046-38902a7386a6 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:12.986758 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:12.990802 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 372/739] 10.222.0.26 () {60 vars in 1141 bytes} [Thu Jul 14 09:05:12 2022] POST /identity/v3/services => generated 251 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 09:05:12.992256 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59012a81-fad2-4267-8d2b-c2f5de90e9a7 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:12.992698 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59012a81-fad2-4267-8d2b-c2f5de90e9a7 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:12.993095 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59012a81-fad2-4267-8d2b-c2f5de90e9a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:13.018241 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-59012a81-fad2-4267-8d2b-c2f5de90e9a7 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:05:13.019963 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-59012a81-fad2-4267-8d2b-c2f5de90e9a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:05:13.047150 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5356dd-50d4-44aa-8af0-daf7fc570ebf None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:13.100486 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0b4a39-46b9-471f-98f6-a26304020500 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:13.101240 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0b4a39-46b9-471f-98f6-a26304020500 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:13.101495 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0b4a39-46b9-471f-98f6-a26304020500 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:13.101833 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0b4a39-46b9-471f-98f6-a26304020500 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:13.102069 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 373/740] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:05:12 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 09:05:13.103546 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59012a81-fad2-4267-8d2b-c2f5de90e9a7 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:13.167097 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:13.178753 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0b4a39-46b9-471f-98f6-a26304020500 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:13.180563 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0b4a39-46b9-471f-98f6-a26304020500 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:13.217629 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59012a81-fad2-4267-8d2b-c2f5de90e9a7 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:13.242112 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:13.254269 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918759c3-23c1-465f-99e3-9983794b0a90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:13.254926 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918759c3-23c1-465f-99e3-9983794b0a90 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:13.255242 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918759c3-23c1-465f-99e3-9983794b0a90 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:13.255516 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918759c3-23c1-465f-99e3-9983794b0a90 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:13.292200 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 368/741] 10.222.0.26 () {58 vars in 1282 bytes} [Thu Jul 14 09:05:12 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 09:05:13.293036 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0b4a39-46b9-471f-98f6-a26304020500 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:13.336016 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918759c3-23c1-465f-99e3-9983794b0a90 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:13.337611 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918759c3-23c1-465f-99e3-9983794b0a90 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:13.400385 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:13.408131 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0b4a39-46b9-471f-98f6-a26304020500 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:13.446171 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5548ed3b-b1db-4c2a-bb5f-6b9192bfa862 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:13.446758 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5548ed3b-b1db-4c2a-bb5f-6b9192bfa862 None admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:13.447060 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5548ed3b-b1db-4c2a-bb5f-6b9192bfa862 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:13.447359 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5548ed3b-b1db-4c2a-bb5f-6b9192bfa862 None admin] PATH_INFO: `/v3/services` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:13.448077 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5548ed3b-b1db-4c2a-bb5f-6b9192bfa862 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:13.449549 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5548ed3b-b1db-4c2a-bb5f-6b9192bfa862 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:13.451774 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 374/742] 10.222.0.26 () {58 vars in 1282 bytes} [Thu Jul 14 09:05:13 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 09:05:13.452986 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb44807-f755-46db-95c4-3dbb075f2d25 None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:13.453287 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb44807-f755-46db-95c4-3dbb075f2d25 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:13.453567 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb44807-f755-46db-95c4-3dbb075f2d25 None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:13.455327 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 375/743] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:05:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:05:13.455755 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb44807-f755-46db-95c4-3dbb075f2d25 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:13.466402 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 369/744] 10.222.0.26 () {60 vars in 1141 bytes} [Thu Jul 14 09:05:13 2022] POST /identity/v3/services => generated 240 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 09:05:13.467155 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5548ed3b-b1db-4c2a-bb5f-6b9192bfa862 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:13.485889 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:13.599694 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb44807-f755-46db-95c4-3dbb075f2d25 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:13.609412 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5548ed3b-b1db-4c2a-bb5f-6b9192bfa862 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:13.640295 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9a8911-0943-4583-9858-b2f67dd0804d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:13.641452 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9a8911-0943-4583-9858-b2f67dd0804d None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:13.641903 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9a8911-0943-4583-9858-b2f67dd0804d None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:13.642362 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9a8911-0943-4583-9858-b2f67dd0804d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:13.653228 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9a8911-0943-4583-9858-b2f67dd0804d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:13.654724 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9a8911-0943-4583-9858-b2f67dd0804d None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:13.654975 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb056db-fe3f-4a8b-869c-5be9a3bee2fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:13.656265 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb056db-fe3f-4a8b-869c-5be9a3bee2fd None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:13.656265 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb056db-fe3f-4a8b-869c-5be9a3bee2fd None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:13.658735 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb056db-fe3f-4a8b-869c-5be9a3bee2fd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:13.673966 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb056db-fe3f-4a8b-869c-5be9a3bee2fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:13.675458 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb056db-fe3f-4a8b-869c-5be9a3bee2fd None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:13.680033 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-0e9a8911-0943-4583-9858-b2f67dd0804d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 09:05:13.680033 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:05:13.680033 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:05:13.680033 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:05:13.680033 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:05:13.680033 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:05:13.680033 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:05:13.680033 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:05:13.680033 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:05:13.680033 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:05:13.680033 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:05:13.680033 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:05:13.680033 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 09:05:13.680033 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 09:05:13.680033 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 09:05:13.680919 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 09:05:13.680919 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:05:13.680919 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:05:13.680919 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:05:13.680919 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:05:13.680919 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:05:13.680919 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:05:13.680919 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:05:13.680919 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 14 09:05:13.680919 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:05:13.680919 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:05:13.680919 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:05:13.680919 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:05:13.680919 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:05:13.680919 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:05:13.680919 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:05:13.680919 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:05:13.681820 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 09:05:13.681820 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 09:05:13.681820 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 09:05:13.681820 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 14 09:05:13.681820 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 376/745] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jul 14 09:05:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:05:13.683088 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d5d2c3-c1ed-44f3-8794-6327f33fd57b None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:13.683088 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d5d2c3-c1ed-44f3-8794-6327f33fd57b None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:13.683372 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d5d2c3-c1ed-44f3-8794-6327f33fd57b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:13.694241 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-6bb056db-fe3f-4a8b-869c-5be9a3bee2fd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 09:05:13.694241 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:05:13.694241 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:05:13.694241 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:05:13.694241 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:05:13.694241 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:05:13.694241 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:05:13.694241 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:05:13.694241 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:05:13.694241 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:05:13.694241 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:05:13.694241 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:05:13.694241 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 09:05:13.694241 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 09:05:13.694241 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 09:05:13.696093 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 09:05:13.696093 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:05:13.696093 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:05:13.696093 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:05:13.696093 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:05:13.696093 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:05:13.696093 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:05:13.696093 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:05:13.696093 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 14 09:05:13.696093 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:05:13.696093 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:05:13.696093 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:05:13.696093 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:05:13.696093 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:05:13.696093 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:05:13.696093 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:05:13.696093 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:05:13.698777 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 09:05:13.698777 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 09:05:13.698777 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 09:05:13.698777 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 14 09:05:13.698777 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 370/746] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jul 14 09:05:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:05:13.698777 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb056db-fe3f-4a8b-869c-5be9a3bee2fd None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:13.699943 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:13.712740 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-25d5d2c3-c1ed-44f3-8794-6327f33fd57b None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:05:13.714245 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-25d5d2c3-c1ed-44f3-8794-6327f33fd57b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:05:13.774667 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:13.798687 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 377/747] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:05:13 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 09:05:13.800025 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d5d2c3-c1ed-44f3-8794-6327f33fd57b None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:13.811457 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb056db-fe3f-4a8b-869c-5be9a3bee2fd None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:13.858696 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0baccb1-5de6-4cca-81ab-06366c57d552 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:13.859427 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0baccb1-5de6-4cca-81ab-06366c57d552 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:13.859716 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0baccb1-5de6-4cca-81ab-06366c57d552 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:13.859994 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0baccb1-5de6-4cca-81ab-06366c57d552 None admin] PATH_INFO: `/v3/domains` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:13.860729 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0baccb1-5de6-4cca-81ab-06366c57d552 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:05:13.861037 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0baccb1-5de6-4cca-81ab-06366c57d552 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:13.862951 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0baccb1-5de6-4cca-81ab-06366c57d552 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:13.881721 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 371/748] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Jul 14 09:05:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:05:13.882654 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0baccb1-5de6-4cca-81ab-06366c57d552 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:13.922806 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d5d2c3-c1ed-44f3-8794-6327f33fd57b None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:13.965377 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1718b395-5e64-423d-a8db-8cf00304a94b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:13.966026 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1718b395-5e64-423d-a8db-8cf00304a94b None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:13.966552 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1718b395-5e64-423d-a8db-8cf00304a94b None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:13.966552 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1718b395-5e64-423d-a8db-8cf00304a94b None admin] PATH_INFO: `/v3/domains` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:13.967362 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1718b395-5e64-423d-a8db-8cf00304a94b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:05:13.967676 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1718b395-5e64-423d-a8db-8cf00304a94b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:13.969123 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1718b395-5e64-423d-a8db-8cf00304a94b None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:13.972089 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:13.985892 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 378/749] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Jul 14 09:05:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:05:13.986806 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1718b395-5e64-423d-a8db-8cf00304a94b None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:14.108582 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1718b395-5e64-423d-a8db-8cf00304a94b None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:14.131052 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0baccb1-5de6-4cca-81ab-06366c57d552 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:14.147876 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109ca09a-11fc-4185-ac7b-e5dd0a81434f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:14.148823 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109ca09a-11fc-4185-ac7b-e5dd0a81434f None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:14.149102 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109ca09a-11fc-4185-ac7b-e5dd0a81434f None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:14.149465 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109ca09a-11fc-4185-ac7b-e5dd0a81434f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:14.156948 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109ca09a-11fc-4185-ac7b-e5dd0a81434f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:14.158681 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109ca09a-11fc-4185-ac7b-e5dd0a81434f None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:14.163580 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:14.175503 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-109ca09a-11fc-4185-ac7b-e5dd0a81434f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 09:05:14.175503 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:05:14.175503 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:05:14.175503 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:05:14.175503 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:05:14.175503 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:05:14.175503 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:05:14.175503 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:05:14.175503 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:05:14.175503 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:05:14.175503 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:05:14.175503 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:05:14.175503 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 09:05:14.175503 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 09:05:14.175503 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 09:05:14.176641 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 09:05:14.176641 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:05:14.176641 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:05:14.176641 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:05:14.176641 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:05:14.176641 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:05:14.176641 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:05:14.176641 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:05:14.176641 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 14 09:05:14.176641 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:05:14.176641 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:05:14.176641 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:05:14.176641 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:05:14.176641 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:05:14.176641 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:05:14.176641 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:05:14.176641 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:05:14.177728 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 09:05:14.177728 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 09:05:14.177728 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 09:05:14.177728 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 09:05:14.177728 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 09:05:14.177728 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 09:05:14.177728 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 09:05:14.177728 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 14 09:05:14.178694 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bebc71c-4143-4f97-82cb-255810730666 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:14.179809 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bebc71c-4143-4f97-82cb-255810730666 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:14.180370 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bebc71c-4143-4f97-82cb-255810730666 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:14.180822 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bebc71c-4143-4f97-82cb-255810730666 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:14.183277 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 379/750] 10.222.0.26 () {56 vars in 1111 bytes} [Thu Jul 14 09:05:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:05:14.184312 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109ca09a-11fc-4185-ac7b-e5dd0a81434f None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:14.199169 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bebc71c-4143-4f97-82cb-255810730666 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 09:05:14.209652 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-8bebc71c-4143-4f97-82cb-255810730666 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 09:05:14.209652 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:05:14.209652 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:05:14.209652 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:05:14.209652 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:05:14.209652 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:05:14.209652 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:05:14.209652 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:05:14.209652 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:05:14.209652 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:05:14.209652 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:05:14.209652 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:05:14.209652 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 09:05:14.209652 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 09:05:14.209652 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 09:05:14.209652 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 09:05:14.210771 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 09:05:14.210771 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 09:05:14.210771 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 09:05:14.210771 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 09:05:14.210771 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:05:14.210771 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:05:14.210771 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:05:14.210771 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:05:14.210771 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:05:14.210771 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:05:14.210771 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:05:14.210771 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 14 09:05:14.210771 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:05:14.210771 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:05:14.210771 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:05:14.210771 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:05:14.211676 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:05:14.211676 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:05:14.211676 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:05:14.211676 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:05:14.211676 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 09:05:14.211676 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 09:05:14.211676 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 09:05:14.211676 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 09:05:14.211676 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 09:05:14.211676 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 09:05:14.211676 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 09:05:14.211676 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 14 09:05:14.211676 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 372/751] 10.222.0.26 () {56 vars in 1096 bytes} [Thu Jul 14 09:05:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 329 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:05:14.212371 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bebc71c-4143-4f97-82cb-255810730666 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:14.251248 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:14.301756 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109ca09a-11fc-4185-ac7b-e5dd0a81434f None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:14.339516 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db40eea-920c-4807-a679-6e0a49e55f86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:14.340994 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db40eea-920c-4807-a679-6e0a49e55f86 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:14.341642 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db40eea-920c-4807-a679-6e0a49e55f86 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:14.342309 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db40eea-920c-4807-a679-6e0a49e55f86 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:14.344481 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bebc71c-4143-4f97-82cb-255810730666 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:14.350463 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db40eea-920c-4807-a679-6e0a49e55f86 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:14.352220 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db40eea-920c-4807-a679-6e0a49e55f86 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:14.367131 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-9db40eea-920c-4807-a679-6e0a49e55f86 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 09:05:14.367131 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:05:14.367131 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:05:14.367131 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:05:14.367131 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:05:14.367131 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:05:14.367131 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:05:14.367131 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:05:14.367131 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:05:14.367131 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:05:14.367131 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:05:14.367131 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:05:14.367131 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 09:05:14.367131 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 09:05:14.367131 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 09:05:14.368214 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 09:05:14.368214 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:05:14.368214 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:05:14.368214 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:05:14.368214 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:05:14.368214 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:05:14.368214 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:05:14.368214 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:05:14.368214 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 14 09:05:14.368214 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:05:14.368214 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:05:14.368214 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:05:14.368214 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:05:14.368214 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:05:14.368214 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:05:14.368214 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:05:14.368214 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:05:14.369271 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 09:05:14.369271 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 09:05:14.369271 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 09:05:14.369271 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 09:05:14.369271 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 09:05:14.369271 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 09:05:14.369271 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 09:05:14.369271 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 14 09:05:14.371358 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 380/752] 10.222.0.26 () {56 vars in 1111 bytes} [Thu Jul 14 09:05:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:05:14.372357 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db40eea-920c-4807-a679-6e0a49e55f86 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:14.383901 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f654ecd-3249-4257-82ad-3ff2a6982133 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:14.384579 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f654ecd-3249-4257-82ad-3ff2a6982133 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:14.384863 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f654ecd-3249-4257-82ad-3ff2a6982133 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:14.385141 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f654ecd-3249-4257-82ad-3ff2a6982133 None admin] PATH_INFO: `/v3/projects` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:14.385939 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f654ecd-3249-4257-82ad-3ff2a6982133 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:05:14.386247 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f654ecd-3249-4257-82ad-3ff2a6982133 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:14.388568 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f654ecd-3249-4257-82ad-3ff2a6982133 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:14.404302 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 373/753] 10.222.0.26 () {56 vars in 1148 bytes} [Thu Jul 14 09:05:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:05:14.405353 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f654ecd-3249-4257-82ad-3ff2a6982133 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:14.462598 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:14.482234 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db40eea-920c-4807-a679-6e0a49e55f86 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:14.499203 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:14.519338 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55598aba-2fc7-441c-9e14-1622a786a795 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:14.520294 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55598aba-2fc7-441c-9e14-1622a786a795 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:14.520760 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55598aba-2fc7-441c-9e14-1622a786a795 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:14.521216 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55598aba-2fc7-441c-9e14-1622a786a795 None admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:14.521767 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f654ecd-3249-4257-82ad-3ff2a6982133 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:14.522400 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55598aba-2fc7-441c-9e14-1622a786a795 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:05:14.522886 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55598aba-2fc7-441c-9e14-1622a786a795 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:14.524481 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55598aba-2fc7-441c-9e14-1622a786a795 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:14.533741 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 381/754] 10.222.0.26 () {56 vars in 1099 bytes} [Thu Jul 14 09:05:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:05:14.534827 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55598aba-2fc7-441c-9e14-1622a786a795 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:14.559978 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68eff69-1d2c-4d49-bd4a-c1f538202dc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:14.560817 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68eff69-1d2c-4d49-bd4a-c1f538202dc7 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:14.561139 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68eff69-1d2c-4d49-bd4a-c1f538202dc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:14.561564 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68eff69-1d2c-4d49-bd4a-c1f538202dc7 None admin] PATH_INFO: `/v3/projects` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:14.562583 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68eff69-1d2c-4d49-bd4a-c1f538202dc7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:05:14.562957 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68eff69-1d2c-4d49-bd4a-c1f538202dc7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:14.564707 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68eff69-1d2c-4d49-bd4a-c1f538202dc7 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:14.578018 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 374/755] 10.222.0.26 () {56 vars in 1148 bytes} [Thu Jul 14 09:05:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:05:14.579368 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68eff69-1d2c-4d49-bd4a-c1f538202dc7 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:14.641545 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55598aba-2fc7-441c-9e14-1622a786a795 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:14.678279 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5052a57-1566-4d32-8e0f-a88a29df5837 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:14.679344 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5052a57-1566-4d32-8e0f-a88a29df5837 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:14.679814 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5052a57-1566-4d32-8e0f-a88a29df5837 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:14.680300 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5052a57-1566-4d32-8e0f-a88a29df5837 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:14.681325 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5052a57-1566-4d32-8e0f-a88a29df5837 None admin] RBAC: Adding query filter params (role.id=7b549d58d1724395b343609d895c80d0, scope.project.id=33bf210d015847439655b9943d1b0261, user.id=e5a43c80514b4a13af03c7f8ebcf7a5f) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:05:14.681869 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5052a57-1566-4d32-8e0f-a88a29df5837 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:14.683635 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5052a57-1566-4d32-8e0f-a88a29df5837 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:14.690120 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68eff69-1d2c-4d49-bd4a-c1f538202dc7 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:14.690703 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 382/756] 10.222.0.26 () {56 vars in 1374 bytes} [Thu Jul 14 09:05:14 2022] GET /identity/v3/role_assignments?user.id=e5a43c80514b4a13af03c7f8ebcf7a5f&scope.project.id=33bf210d015847439655b9943d1b0261&role.id=7b549d58d1724395b343609d895c80d0 => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:05:14.691737 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5052a57-1566-4d32-8e0f-a88a29df5837 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:14.698969 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:14.732577 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3672ee94-6963-4382-a597-eb2ae406a762 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:14.733383 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3672ee94-6963-4382-a597-eb2ae406a762 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:14.733732 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3672ee94-6963-4382-a597-eb2ae406a762 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:14.734075 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3672ee94-6963-4382-a597-eb2ae406a762 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:14.742129 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:14.746588 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3672ee94-6963-4382-a597-eb2ae406a762 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:14.749138 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3672ee94-6963-4382-a597-eb2ae406a762 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:14.771206 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-3672ee94-6963-4382-a597-eb2ae406a762 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 09:05:14.771206 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:05:14.771206 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:05:14.771206 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:05:14.771206 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:05:14.771206 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:05:14.771206 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:05:14.771206 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:05:14.771206 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:05:14.771206 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:05:14.771206 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:05:14.771206 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:05:14.771206 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 09:05:14.771206 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 09:05:14.771206 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 09:05:14.772334 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 09:05:14.772334 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:05:14.772334 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:05:14.772334 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:05:14.772334 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:05:14.772334 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:05:14.772334 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:05:14.772334 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:05:14.772334 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 14 09:05:14.772334 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:05:14.772334 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:05:14.772334 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:05:14.772334 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:05:14.772334 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:05:14.772334 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:05:14.772334 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:05:14.772334 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:05:14.773619 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 09:05:14.773619 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 09:05:14.773619 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 09:05:14.773619 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 14 09:05:14.773619 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 375/757] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jul 14 09:05:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:05:14.784386 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3672ee94-6963-4382-a597-eb2ae406a762 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:14.813138 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5052a57-1566-4d32-8e0f-a88a29df5837 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:14.850042 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134412a9-4c0a-412b-af27-c07637cee149 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:14.851213 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134412a9-4c0a-412b-af27-c07637cee149 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:14.851705 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134412a9-4c0a-412b-af27-c07637cee149 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:14.852196 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134412a9-4c0a-412b-af27-c07637cee149 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:14.853260 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134412a9-4c0a-412b-af27-c07637cee149 None admin] RBAC: Adding query filter params (role.id=f5c487bc605f45df82ef6367c80fae70, scope.project.id=33bf210d015847439655b9943d1b0261, user.id=7d6267ede8d64e2abe1f4058a9797fb7) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:05:14.853788 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134412a9-4c0a-412b-af27-c07637cee149 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:14.855586 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134412a9-4c0a-412b-af27-c07637cee149 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:14.862595 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 383/758] 10.222.0.26 () {56 vars in 1374 bytes} [Thu Jul 14 09:05:14 2022] GET /identity/v3/role_assignments?user.id=7d6267ede8d64e2abe1f4058a9797fb7&scope.project.id=33bf210d015847439655b9943d1b0261&role.id=f5c487bc605f45df82ef6367c80fae70 => generated 259 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:05:14.930198 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3672ee94-6963-4382-a597-eb2ae406a762 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:14.966345 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfb2c79-a646-44af-a802-43d7a93a95d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:14.967034 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfb2c79-a646-44af-a802-43d7a93a95d5 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:14.967352 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfb2c79-a646-44af-a802-43d7a93a95d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:14.967629 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfb2c79-a646-44af-a802-43d7a93a95d5 None admin] PATH_INFO: `/v3/domains` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:14.968379 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfb2c79-a646-44af-a802-43d7a93a95d5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:05:14.968686 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfb2c79-a646-44af-a802-43d7a93a95d5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:14.970129 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfb2c79-a646-44af-a802-43d7a93a95d5 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:14.984097 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 376/759] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Jul 14 09:05:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:05:14.989424 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134412a9-4c0a-412b-af27-c07637cee149 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:15.133356 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134412a9-4c0a-412b-af27-c07637cee149 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:15.140841 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:15.177980 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af02fc52-b177-4e63-bd86-f1b449a20bb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:15.178909 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af02fc52-b177-4e63-bd86-f1b449a20bb8 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:15.179234 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af02fc52-b177-4e63-bd86-f1b449a20bb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:15.179524 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af02fc52-b177-4e63-bd86-f1b449a20bb8 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:15.187765 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af02fc52-b177-4e63-bd86-f1b449a20bb8 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:15.189229 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af02fc52-b177-4e63-bd86-f1b449a20bb8 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:15.210635 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-af02fc52-b177-4e63-bd86-f1b449a20bb8 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 14 09:05:15.210635 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:05:15.210635 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:05:15.210635 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:05:15.210635 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:05:15.210635 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:05:15.210635 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:05:15.210635 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:05:15.210635 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:05:15.210635 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:05:15.210635 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:05:15.210635 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:05:15.210635 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 09:05:15.210635 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 09:05:15.210635 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 09:05:15.210635 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 09:05:15.211699 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:05:15.211699 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:05:15.211699 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 09:05:15.211699 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 09:05:15.211699 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 09:05:15.211699 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 09:05:15.211699 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:05:15.211699 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:05:15.211699 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:05:15.211699 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:05:15.211699 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:05:15.211699 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 14 09:05:15.211699 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:05:15.211699 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:05:15.211699 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:05:15.211699 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:05:15.211699 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:05:15.211699 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:05:15.212672 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:05:15.212672 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:05:15.212672 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 09:05:15.212672 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 09:05:15.212672 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 09:05:15.212672 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 09:05:15.212672 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 09:05:15.212672 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 09:05:15.212672 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 14 09:05:15.212672 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 14 09:05:15.215911 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 384/760] 10.222.0.26 () {56 vars in 1093 bytes} [Thu Jul 14 09:05:14 2022] GET /identity/v3/users/nova => generated 82 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:05:15.221353 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfb2c79-a646-44af-a802-43d7a93a95d5 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:15.258263 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:15.371954 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfb2c79-a646-44af-a802-43d7a93a95d5 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:15.381189 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafb27a3-2d6d-427a-ac25-4b53f1d547ca None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:15.381538 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafb27a3-2d6d-427a-ac25-4b53f1d547ca None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:15.381827 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafb27a3-2d6d-427a-ac25-4b53f1d547ca None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:15.383250 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 385/761] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:05:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:05:15.400832 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e628ace-4c49-4463-9ac6-6899f3d35b35 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:15.401012 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e628ace-4c49-4463-9ac6-6899f3d35b35 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:15.401293 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e628ace-4c49-4463-9ac6-6899f3d35b35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:15.417968 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33e8471-ebaa-4453-be70-20e3b8491e8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:15.419959 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33e8471-ebaa-4453-be70-20e3b8491e8a None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:15.420596 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33e8471-ebaa-4453-be70-20e3b8491e8a None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:15.421122 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33e8471-ebaa-4453-be70-20e3b8491e8a None admin] PATH_INFO: `/v3/users` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:15.422195 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33e8471-ebaa-4453-be70-20e3b8491e8a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:05:15.422710 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33e8471-ebaa-4453-be70-20e3b8491e8a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:15.424412 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33e8471-ebaa-4453-be70-20e3b8491e8a None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:15.431486 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-8e628ace-4c49-4463-9ac6-6899f3d35b35 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:05:15.432684 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-8e628ace-4c49-4463-9ac6-6899f3d35b35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:05:15.453789 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 377/762] 10.222.0.26 () {56 vars in 1133 bytes} [Thu Jul 14 09:05:15 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:05:15.459403 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33e8471-ebaa-4453-be70-20e3b8491e8a None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:15.560650 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 386/763] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:05:15 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 09:05:15.592196 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:15.618934 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33e8471-ebaa-4453-be70-20e3b8491e8a None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:15.629937 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e628ace-4c49-4463-9ac6-6899f3d35b35 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:15.689315 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50cca53-fae6-4d30-8304-3f451d0fe518 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:15.690034 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50cca53-fae6-4d30-8304-3f451d0fe518 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:15.690315 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50cca53-fae6-4d30-8304-3f451d0fe518 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:15.690843 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50cca53-fae6-4d30-8304-3f451d0fe518 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:15.706658 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50cca53-fae6-4d30-8304-3f451d0fe518 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:15.708145 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50cca53-fae6-4d30-8304-3f451d0fe518 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:15.727415 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-b50cca53-fae6-4d30-8304-3f451d0fe518 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 09:05:15.727415 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:05:15.727415 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:05:15.727415 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:05:15.727415 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:05:15.727415 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:05:15.727415 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:05:15.727415 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:05:15.727415 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:05:15.727415 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:05:15.727415 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:05:15.727415 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:05:15.727415 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 09:05:15.727415 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 09:05:15.727415 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 09:05:15.728377 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 09:05:15.728377 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:05:15.728377 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:05:15.728377 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:05:15.728377 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:05:15.728377 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:05:15.728377 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:05:15.728377 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:05:15.728377 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 14 09:05:15.728377 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:05:15.728377 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:05:15.728377 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:05:15.728377 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:05:15.728377 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:05:15.728377 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:05:15.728377 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:05:15.728377 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:05:15.729281 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 09:05:15.729281 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 09:05:15.729281 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 09:05:15.729281 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 14 09:05:15.733219 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 378/764] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jul 14 09:05:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 09:05:15.734472 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50cca53-fae6-4d30-8304-3f451d0fe518 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:15.797509 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:15.808035 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e628ace-4c49-4463-9ac6-6899f3d35b35 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:15.851110 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3170bf12-32d3-48bb-85aa-0b3a8ef7d5c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:15.851968 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3170bf12-32d3-48bb-85aa-0b3a8ef7d5c9 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:15.852250 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3170bf12-32d3-48bb-85aa-0b3a8ef7d5c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:15.852523 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3170bf12-32d3-48bb-85aa-0b3a8ef7d5c9 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:15.853188 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3170bf12-32d3-48bb-85aa-0b3a8ef7d5c9 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:15.855378 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3170bf12-32d3-48bb-85aa-0b3a8ef7d5c9 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:15.866365 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50cca53-fae6-4d30-8304-3f451d0fe518 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:15.869735 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-3170bf12-32d3-48bb-85aa-0b3a8ef7d5c9 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 14 09:05:15.869735 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:05:15.869735 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 09:05:15.869735 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 09:05:15.869735 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 09:05:15.869735 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 09:05:15.869735 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 09:05:15.869735 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 09:05:15.869735 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 14 09:05:15.869735 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 14 09:05:15.869735 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 09:05:15.869735 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 14 09:05:15.869735 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:05:15.869735 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:05:15.869735 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:05:15.869735 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:05:15.870666 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:05:15.870666 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:05:15.870666 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:05:15.870666 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:05:15.870666 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:05:15.870666 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:05:15.870666 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:05:15.870666 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 09:05:15.870666 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 09:05:15.870666 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 09:05:15.870666 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 09:05:15.870666 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:05:15.870666 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:05:15.870666 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:05:15.870666 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:05:15.870666 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:05:15.871550 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:05:15.871550 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:05:15.871550 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 14 09:05:15.871550 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:05:15.871550 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:05:15.871550 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:05:15.871550 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:05:15.871550 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:05:15.871550 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:05:15.871550 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:05:15.871550 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:05:15.871550 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 09:05:15.871550 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 09:05:15.871550 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 14 09:05:15.871550 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 14 09:05:15.876279 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 387/765] 10.222.0.26 () {56 vars in 1129 bytes} [Thu Jul 14 09:05:15 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 09:05:15.877590 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3083139b-df96-4047-8c57-878f9c7eb4c1 None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:15.877880 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3083139b-df96-4047-8c57-878f9c7eb4c1 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:15.878160 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3083139b-df96-4047-8c57-878f9c7eb4c1 None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:15.883949 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 388/766] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:05:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:05:15.884847 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3083139b-df96-4047-8c57-878f9c7eb4c1 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:15.898524 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:15.908168 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5541fd5f-76f6-41e0-8c34-0c6d8609371e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:15.909231 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5541fd5f-76f6-41e0-8c34-0c6d8609371e None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:15.909516 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5541fd5f-76f6-41e0-8c34-0c6d8609371e None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:15.909785 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5541fd5f-76f6-41e0-8c34-0c6d8609371e None admin] PATH_INFO: `/v3/domains` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:15.910524 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5541fd5f-76f6-41e0-8c34-0c6d8609371e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:05:15.910832 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5541fd5f-76f6-41e0-8c34-0c6d8609371e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:15.912771 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5541fd5f-76f6-41e0-8c34-0c6d8609371e None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:15.931227 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 379/767] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Jul 14 09:05:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 09:05:15.931227 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49a8c4a-6159-446c-88e6-961af74ac470 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:15.931227 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49a8c4a-6159-446c-88e6-961af74ac470 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:15.931227 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49a8c4a-6159-446c-88e6-961af74ac470 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:15.951470 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-d49a8c4a-6159-446c-88e6-961af74ac470 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:05:15.953104 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-d49a8c4a-6159-446c-88e6-961af74ac470 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:05:16.013266 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3083139b-df96-4047-8c57-878f9c7eb4c1 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:16.032251 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 380/768] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:05:15 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 09:05:16.033539 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49a8c4a-6159-446c-88e6-961af74ac470 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:16.068971 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52091ed5-378c-4db8-ad44-8c9c721fbb87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:16.070063 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52091ed5-378c-4db8-ad44-8c9c721fbb87 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:16.070567 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52091ed5-378c-4db8-ad44-8c9c721fbb87 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:16.071039 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52091ed5-378c-4db8-ad44-8c9c721fbb87 None admin] PATH_INFO: `/v3/services` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:16.072110 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52091ed5-378c-4db8-ad44-8c9c721fbb87 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:05:16.072699 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52091ed5-378c-4db8-ad44-8c9c721fbb87 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:16.074333 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52091ed5-378c-4db8-ad44-8c9c721fbb87 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:16.084661 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 389/769] 10.222.0.26 () {56 vars in 1124 bytes} [Thu Jul 14 09:05:15 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:05:16.085771 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52091ed5-378c-4db8-ad44-8c9c721fbb87 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:16.117054 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:16.170047 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49a8c4a-6159-446c-88e6-961af74ac470 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:16.192851 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:16.199776 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52091ed5-378c-4db8-ad44-8c9c721fbb87 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:16.217461 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3738d7-9800-4c17-84f5-e3e3e2509411 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:16.218198 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3738d7-9800-4c17-84f5-e3e3e2509411 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:16.218481 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3738d7-9800-4c17-84f5-e3e3e2509411 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:16.218756 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3738d7-9800-4c17-84f5-e3e3e2509411 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:16.227021 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3738d7-9800-4c17-84f5-e3e3e2509411 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:16.228536 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3738d7-9800-4c17-84f5-e3e3e2509411 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:16.243918 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-7a3738d7-9800-4c17-84f5-e3e3e2509411 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 09:05:16.243918 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:05:16.243918 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:05:16.243918 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:05:16.243918 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:05:16.243918 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:05:16.243918 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:05:16.243918 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:05:16.243918 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:05:16.243918 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:05:16.243918 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:05:16.243918 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:05:16.243918 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 09:05:16.243918 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 09:05:16.243918 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 09:05:16.244909 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 09:05:16.244909 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:05:16.244909 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:05:16.244909 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:05:16.244909 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:05:16.244909 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:05:16.244909 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:05:16.244909 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:05:16.244909 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 14 09:05:16.244909 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:05:16.244909 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:05:16.244909 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:05:16.244909 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:05:16.244909 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:05:16.244909 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:05:16.244909 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:05:16.244909 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:05:16.246321 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 09:05:16.246321 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 09:05:16.246321 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 09:05:16.246321 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 09:05:16.246321 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 09:05:16.246321 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 09:05:16.246321 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 09:05:16.246321 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 14 09:05:16.246321 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 381/770] 10.222.0.26 () {56 vars in 1111 bytes} [Thu Jul 14 09:05:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:05:16.246321 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3738d7-9800-4c17-84f5-e3e3e2509411 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:16.362843 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306c11f2-061a-414c-aa05-0f1b20fb4950 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:16.363758 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306c11f2-061a-414c-aa05-0f1b20fb4950 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:16.364038 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306c11f2-061a-414c-aa05-0f1b20fb4950 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:16.364398 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306c11f2-061a-414c-aa05-0f1b20fb4950 None admin] PATH_INFO: `/v3/services/network` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:16.365096 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306c11f2-061a-414c-aa05-0f1b20fb4950 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:16.366536 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306c11f2-061a-414c-aa05-0f1b20fb4950 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:16.367344 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3738d7-9800-4c17-84f5-e3e3e2509411 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:16.378942 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-306c11f2-061a-414c-aa05-0f1b20fb4950 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 14 09:05:16.378942 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:05:16.378942 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 09:05:16.378942 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 09:05:16.378942 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 09:05:16.378942 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 09:05:16.378942 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 09:05:16.378942 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 09:05:16.378942 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 14 09:05:16.378942 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 14 09:05:16.378942 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 09:05:16.378942 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 14 09:05:16.378942 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:05:16.378942 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:05:16.378942 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:05:16.378942 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:05:16.378942 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:05:16.381311 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:05:16.381311 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:05:16.381311 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:05:16.381311 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:05:16.381311 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:05:16.381311 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:05:16.381311 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 09:05:16.381311 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 09:05:16.381311 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 09:05:16.381311 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 09:05:16.381311 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:05:16.381311 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:05:16.381311 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:05:16.381311 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:05:16.381311 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:05:16.381311 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:05:16.383355 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:05:16.383355 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 14 09:05:16.383355 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:05:16.383355 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:05:16.383355 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:05:16.383355 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:05:16.383355 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:05:16.383355 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:05:16.383355 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:05:16.383355 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:05:16.383355 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 09:05:16.383355 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 09:05:16.383355 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 14 09:05:16.383355 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 14 09:05:16.383355 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 390/771] 10.222.0.26 () {56 vars in 1111 bytes} [Thu Jul 14 09:05:16 2022] GET /identity/v3/services/network => generated 88 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:05:16.385112 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306c11f2-061a-414c-aa05-0f1b20fb4950 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:16.415096 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd992e0-391c-4d11-a89b-e0f777b19aaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:16.416064 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd992e0-391c-4d11-a89b-e0f777b19aaf None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:16.416350 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd992e0-391c-4d11-a89b-e0f777b19aaf None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:16.416630 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd992e0-391c-4d11-a89b-e0f777b19aaf None admin] PATH_INFO: `/v3/services` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:16.417593 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd992e0-391c-4d11-a89b-e0f777b19aaf None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:05:16.418303 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd992e0-391c-4d11-a89b-e0f777b19aaf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:16.419946 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd992e0-391c-4d11-a89b-e0f777b19aaf None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:16.426559 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 382/772] 10.222.0.26 () {56 vars in 1124 bytes} [Thu Jul 14 09:05:16 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:05:16.427893 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd992e0-391c-4d11-a89b-e0f777b19aaf None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:16.482761 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:16.487104 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:16.495930 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306c11f2-061a-414c-aa05-0f1b20fb4950 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:16.550552 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a0114d-42ed-414f-8172-f493b1c233e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:16.552506 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a0114d-42ed-414f-8172-f493b1c233e4 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:16.552787 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a0114d-42ed-414f-8172-f493b1c233e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:16.553200 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a0114d-42ed-414f-8172-f493b1c233e4 None admin] PATH_INFO: `/v3/projects` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:16.554009 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a0114d-42ed-414f-8172-f493b1c233e4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:05:16.554317 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a0114d-42ed-414f-8172-f493b1c233e4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:16.555789 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a0114d-42ed-414f-8172-f493b1c233e4 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:16.567339 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd992e0-391c-4d11-a89b-e0f777b19aaf None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:16.572967 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 391/773] 10.222.0.26 () {56 vars in 1148 bytes} [Thu Jul 14 09:05:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:05:16.573859 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a0114d-42ed-414f-8172-f493b1c233e4 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:16.612786 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2181db5-d380-4f4e-a6dc-16cd820b3e06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:16.613843 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2181db5-d380-4f4e-a6dc-16cd820b3e06 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:16.614318 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2181db5-d380-4f4e-a6dc-16cd820b3e06 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:16.615307 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2181db5-d380-4f4e-a6dc-16cd820b3e06 None admin] PATH_INFO: `/v3/services` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:16.616279 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2181db5-d380-4f4e-a6dc-16cd820b3e06 None admin] RBAC: Adding query filter params (name=network) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:05:16.616779 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2181db5-d380-4f4e-a6dc-16cd820b3e06 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:16.619135 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2181db5-d380-4f4e-a6dc-16cd820b3e06 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:16.626614 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 383/774] 10.222.0.26 () {56 vars in 1112 bytes} [Thu Jul 14 09:05:16 2022] GET /identity/v3/services?name=network => generated 124 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:05:16.627681 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2181db5-d380-4f4e-a6dc-16cd820b3e06 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:16.680556 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a0114d-42ed-414f-8172-f493b1c233e4 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:16.716657 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917f4c97-7318-4ecd-a895-fad82fee48a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:16.717602 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917f4c97-7318-4ecd-a895-fad82fee48a9 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:16.717938 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917f4c97-7318-4ecd-a895-fad82fee48a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:16.718217 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917f4c97-7318-4ecd-a895-fad82fee48a9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:16.719640 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917f4c97-7318-4ecd-a895-fad82fee48a9 None admin] RBAC: Adding query filter params (interface=public, service_id=ce621fb78eba4d7395b920e87c549377, region_id=RegionOne) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:05:16.719966 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917f4c97-7318-4ecd-a895-fad82fee48a9 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:16.721721 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917f4c97-7318-4ecd-a895-fad82fee48a9 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:16.735463 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 392/775] 10.222.0.26 () {56 vars in 1251 bytes} [Thu Jul 14 09:05:16 2022] GET /identity/v3/endpoints?service_id=ce621fb78eba4d7395b920e87c549377&interface=public®ion_id=RegionOne => generated 194 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:05:16.735463 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917f4c97-7318-4ecd-a895-fad82fee48a9 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:16.751302 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:16.770384 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2181db5-d380-4f4e-a6dc-16cd820b3e06 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:16.778532 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:16.817580 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd652446-9194-4362-8761-2c9aaf5b1895 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:16.820017 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd652446-9194-4362-8761-2c9aaf5b1895 None admin] REQUEST_METHOD: `PUT` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:16.820506 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd652446-9194-4362-8761-2c9aaf5b1895 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:16.820999 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd652446-9194-4362-8761-2c9aaf5b1895 None admin] PATH_INFO: `/v3/projects/33bf210d015847439655b9943d1b0261/users/538fff3c5a2b487eb44ae7c1a1198102/roles/7b549d58d1724395b343609d895c80d0` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:16.823208 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd652446-9194-4362-8761-2c9aaf5b1895 None admin] RBAC: Authorizing `identity:create_grant(project_id=33bf210d015847439655b9943d1b0261, user_id=538fff3c5a2b487eb44ae7c1a1198102, role_id=7b549d58d1724395b343609d895c80d0)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:16.825545 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd652446-9194-4362-8761-2c9aaf5b1895 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:16.843899 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 384/776] 10.222.0.26 () {58 vars in 1439 bytes} [Thu Jul 14 09:05:16 2022] PUT /identity/v3/projects/33bf210d015847439655b9943d1b0261/users/538fff3c5a2b487eb44ae7c1a1198102/roles/7b549d58d1724395b343609d895c80d0 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:05:16.845015 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd652446-9194-4362-8761-2c9aaf5b1895 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:16.867428 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917f4c97-7318-4ecd-a895-fad82fee48a9 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:16.913611 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a5610c-6d52-41b0-993f-fde0de46e27c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:16.915267 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a5610c-6d52-41b0-993f-fde0de46e27c None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:16.915770 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a5610c-6d52-41b0-993f-fde0de46e27c None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:16.916255 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a5610c-6d52-41b0-993f-fde0de46e27c None admin] PATH_INFO: `/v3/services` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:16.917196 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a5610c-6d52-41b0-993f-fde0de46e27c None admin] RBAC: Adding query filter params (type=network) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:05:16.917694 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a5610c-6d52-41b0-993f-fde0de46e27c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:16.920917 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a5610c-6d52-41b0-993f-fde0de46e27c None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:16.929723 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 393/777] 10.222.0.26 () {56 vars in 1112 bytes} [Thu Jul 14 09:05:16 2022] GET /identity/v3/services?type=network => generated 350 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:05:16.932050 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233d52d4-56e6-40b7-b397-f36d1b74b29e None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:16.933019 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233d52d4-56e6-40b7-b397-f36d1b74b29e None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:16.933672 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233d52d4-56e6-40b7-b397-f36d1b74b29e None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:16.939913 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 394/778] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:05:16 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:05:16.941612 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233d52d4-56e6-40b7-b397-f36d1b74b29e None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:16.997231 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd652446-9194-4362-8761-2c9aaf5b1895 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:17.035610 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:17.040543 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e8ce1b-74e8-4af6-8753-5fd9996a829a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:17.041602 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e8ce1b-74e8-4af6-8753-5fd9996a829a None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:17.041885 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e8ce1b-74e8-4af6-8753-5fd9996a829a None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:17.042421 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e8ce1b-74e8-4af6-8753-5fd9996a829a None admin] PATH_INFO: `/v3/services` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:17.043118 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e8ce1b-74e8-4af6-8753-5fd9996a829a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:17.046015 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e8ce1b-74e8-4af6-8753-5fd9996a829a None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:17.053122 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 385/779] 10.222.0.26 () {56 vars in 1087 bytes} [Thu Jul 14 09:05:16 2022] GET /identity/v3/services => generated 772 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:05:17.059772 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812cf08f-44b2-48a7-a231-454dcf4811ef None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:17.060056 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812cf08f-44b2-48a7-a231-454dcf4811ef None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:17.060329 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812cf08f-44b2-48a7-a231-454dcf4811ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:17.072159 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233d52d4-56e6-40b7-b397-f36d1b74b29e None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:17.104437 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-812cf08f-44b2-48a7-a231-454dcf4811ef None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:05:17.105654 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-812cf08f-44b2-48a7-a231-454dcf4811ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:05:17.116814 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c80286-a49a-4316-9bbb-43e39a7536da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:17.117482 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c80286-a49a-4316-9bbb-43e39a7536da None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:17.117761 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c80286-a49a-4316-9bbb-43e39a7536da None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:17.118030 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c80286-a49a-4316-9bbb-43e39a7536da None admin] PATH_INFO: `/v3/endpoints` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:17.118833 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c80286-a49a-4316-9bbb-43e39a7536da None admin] RBAC: Adding query filter params (interface=public, service_id=4df4791a3bf44d8b9c09547d77cf3382, region_id=RegionOne) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:05:17.119139 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c80286-a49a-4316-9bbb-43e39a7536da None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:17.120543 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c80286-a49a-4316-9bbb-43e39a7536da None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:17.127747 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 395/780] 10.222.0.26 () {56 vars in 1251 bytes} [Thu Jul 14 09:05:16 2022] GET /identity/v3/endpoints?service_id=4df4791a3bf44d8b9c09547d77cf3382&interface=public®ion_id=RegionOne => generated 194 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:05:17.132896 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c80286-a49a-4316-9bbb-43e39a7536da None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:17.149793 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:17.214266 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 386/781] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:05:17 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 09:05:17.217321 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c93b6d-ff6d-45e4-a453-802398801181 None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:17.217896 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c93b6d-ff6d-45e4-a453-802398801181 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:17.218405 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c93b6d-ff6d-45e4-a453-802398801181 None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:17.220488 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 387/782] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:05:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:05:17.228720 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea5a1cc-c6ea-4088-b507-14a3cfa45052 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:17.229021 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea5a1cc-c6ea-4088-b507-14a3cfa45052 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:17.229279 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea5a1cc-c6ea-4088-b507-14a3cfa45052 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:17.250322 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-6ea5a1cc-c6ea-4088-b507-14a3cfa45052 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:05:17.251443 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-6ea5a1cc-c6ea-4088-b507-14a3cfa45052 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:05:17.264881 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c80286-a49a-4316-9bbb-43e39a7536da None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:17.311685 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8819f6e4-bbc7-4b63-b6e5-e4bdbbe5f7d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:17.312367 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8819f6e4-bbc7-4b63-b6e5-e4bdbbe5f7d7 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:17.312644 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8819f6e4-bbc7-4b63-b6e5-e4bdbbe5f7d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:17.312912 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8819f6e4-bbc7-4b63-b6e5-e4bdbbe5f7d7 None admin] PATH_INFO: `/v3/services` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:17.313588 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8819f6e4-bbc7-4b63-b6e5-e4bdbbe5f7d7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:17.315082 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8819f6e4-bbc7-4b63-b6e5-e4bdbbe5f7d7 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:17.320470 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 396/783] 10.222.0.26 () {56 vars in 1087 bytes} [Thu Jul 14 09:05:17 2022] GET /identity/v3/services => generated 772 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 09:05:17.321247 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8819f6e4-bbc7-4b63-b6e5-e4bdbbe5f7d7 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:17.333201 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 388/784] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:05:17 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 09:05:17.337304 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:17.362394 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea5a1cc-c6ea-4088-b507-14a3cfa45052 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:17.455271 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8819f6e4-bbc7-4b63-b6e5-e4bdbbe5f7d7 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:17.467235 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:17.479134 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea5a1cc-c6ea-4088-b507-14a3cfa45052 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:17.496353 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ed6ca9-438c-4913-a3ea-6c3321231c55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:17.497567 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ed6ca9-438c-4913-a3ea-6c3321231c55 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:17.498065 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ed6ca9-438c-4913-a3ea-6c3321231c55 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:17.498535 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ed6ca9-438c-4913-a3ea-6c3321231c55 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:17.511339 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ed6ca9-438c-4913-a3ea-6c3321231c55 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 09:05:17.517905 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641b1ab5-64ee-4a10-a2bb-60ba0987e082 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:17.518774 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641b1ab5-64ee-4a10-a2bb-60ba0987e082 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:17.519115 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641b1ab5-64ee-4a10-a2bb-60ba0987e082 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:17.519433 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641b1ab5-64ee-4a10-a2bb-60ba0987e082 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:17.521091 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-56ed6ca9-438c-4913-a3ea-6c3321231c55 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 09:05:17.521091 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:05:17.521091 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:05:17.521091 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:05:17.521091 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:05:17.521091 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:05:17.521091 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:05:17.521091 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:05:17.521091 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:05:17.521091 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:05:17.521091 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:05:17.521091 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:05:17.521091 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 09:05:17.521091 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 09:05:17.521091 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 09:05:17.521091 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 09:05:17.522084 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 09:05:17.522084 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 09:05:17.522084 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 09:05:17.522084 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 09:05:17.522084 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:05:17.522084 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:05:17.522084 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:05:17.522084 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:05:17.522084 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:05:17.522084 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:05:17.522084 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:05:17.522084 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 14 09:05:17.522084 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:05:17.522084 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:05:17.522084 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:05:17.522084 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:05:17.522933 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:05:17.522933 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:05:17.522933 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:05:17.522933 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:05:17.522933 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 09:05:17.522933 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 09:05:17.522933 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 09:05:17.522933 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 09:05:17.522933 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 09:05:17.522933 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 09:05:17.522933 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 09:05:17.522933 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 14 09:05:17.525778 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 397/785] 10.222.0.26 () {56 vars in 1096 bytes} [Thu Jul 14 09:05:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:05:17.537853 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641b1ab5-64ee-4a10-a2bb-60ba0987e082 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 09:05:17.544683 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ed6ca9-438c-4913-a3ea-6c3321231c55 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:17.546680 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-641b1ab5-64ee-4a10-a2bb-60ba0987e082 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 09:05:17.546680 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:05:17.546680 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:05:17.546680 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:05:17.546680 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:05:17.546680 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:05:17.546680 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:05:17.546680 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:05:17.546680 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:05:17.546680 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:05:17.546680 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:05:17.546680 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:05:17.546680 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 09:05:17.546680 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 09:05:17.546680 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 09:05:17.546680 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 09:05:17.547692 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 09:05:17.547692 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 09:05:17.547692 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 09:05:17.547692 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 09:05:17.547692 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:05:17.547692 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:05:17.547692 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:05:17.547692 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:05:17.547692 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:05:17.547692 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:05:17.547692 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:05:17.547692 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 14 09:05:17.547692 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:05:17.547692 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:05:17.547692 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:05:17.547692 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:05:17.548541 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:05:17.548541 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:05:17.548541 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:05:17.548541 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:05:17.548541 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 09:05:17.548541 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 09:05:17.548541 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 09:05:17.548541 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 09:05:17.548541 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 09:05:17.548541 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 09:05:17.548541 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 09:05:17.548541 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 14 09:05:17.548541 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 389/786] 10.222.0.26 () {56 vars in 1102 bytes} [Thu Jul 14 09:05:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:05:17.561281 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641b1ab5-64ee-4a10-a2bb-60ba0987e082 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:17.632689 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:17.689822 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ed6ca9-438c-4913-a3ea-6c3321231c55 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:17.712817 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641b1ab5-64ee-4a10-a2bb-60ba0987e082 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:17.759964 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0282f26-7d8e-451f-ac3c-e47a658c7381 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:17.760642 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0282f26-7d8e-451f-ac3c-e47a658c7381 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:17.760928 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0282f26-7d8e-451f-ac3c-e47a658c7381 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:17.761204 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0282f26-7d8e-451f-ac3c-e47a658c7381 None admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:17.762023 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0282f26-7d8e-451f-ac3c-e47a658c7381 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:05:17.762269 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0282f26-7d8e-451f-ac3c-e47a658c7381 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:17.763693 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0282f26-7d8e-451f-ac3c-e47a658c7381 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:17.775818 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc9c849-94f8-47a4-bb65-dcf13fd33626 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:17.776492 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc9c849-94f8-47a4-bb65-dcf13fd33626 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:17.776781 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc9c849-94f8-47a4-bb65-dcf13fd33626 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:17.777055 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc9c849-94f8-47a4-bb65-dcf13fd33626 None admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:17.777814 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc9c849-94f8-47a4-bb65-dcf13fd33626 None admin] RBAC: Adding query filter params (name=service) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:05:17.778120 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc9c849-94f8-47a4-bb65-dcf13fd33626 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:17.779512 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc9c849-94f8-47a4-bb65-dcf13fd33626 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:17.789198 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 398/787] 10.222.0.26 () {56 vars in 1099 bytes} [Thu Jul 14 09:05:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 09:05:17.790640 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0282f26-7d8e-451f-ac3c-e47a658c7381 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:17.795110 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 390/788] 10.222.0.26 () {56 vars in 1103 bytes} [Thu Jul 14 09:05:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:05:17.801278 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc9c849-94f8-47a4-bb65-dcf13fd33626 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:17.825802 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:17.960629 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0282f26-7d8e-451f-ac3c-e47a658c7381 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:17.986131 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc9c849-94f8-47a4-bb65-dcf13fd33626 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:17.997220 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:18.007523 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a975106d-64fd-4427-b9b0-52c74f7da9d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:18.008509 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a975106d-64fd-4427-b9b0-52c74f7da9d5 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:18.009422 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a975106d-64fd-4427-b9b0-52c74f7da9d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:18.009770 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a975106d-64fd-4427-b9b0-52c74f7da9d5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:18.022828 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a975106d-64fd-4427-b9b0-52c74f7da9d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:18.024937 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a975106d-64fd-4427-b9b0-52c74f7da9d5 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:18.046065 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-a975106d-64fd-4427-b9b0-52c74f7da9d5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 09:05:18.046065 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:05:18.046065 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:05:18.046065 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:05:18.046065 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:05:18.046065 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:05:18.046065 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:05:18.046065 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:05:18.046065 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:05:18.046065 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:05:18.046065 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:05:18.046065 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:05:18.046065 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 09:05:18.046065 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 09:05:18.046065 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 09:05:18.047252 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 09:05:18.047252 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:05:18.047252 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:05:18.047252 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:05:18.047252 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:05:18.047252 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:05:18.047252 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:05:18.047252 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:05:18.047252 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 14 09:05:18.047252 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:05:18.047252 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:05:18.047252 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:05:18.047252 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:05:18.047252 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:05:18.047252 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:05:18.047252 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:05:18.047252 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:05:18.048191 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 09:05:18.048191 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 09:05:18.048191 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 09:05:18.048191 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 14 09:05:18.051564 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 399/789] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jul 14 09:05:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:05:18.054331 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a975106d-64fd-4427-b9b0-52c74f7da9d5 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:18.068437 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7aaaadc-f2e3-4472-b59f-6eff602952a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:18.069279 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7aaaadc-f2e3-4472-b59f-6eff602952a1 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:18.069557 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7aaaadc-f2e3-4472-b59f-6eff602952a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:18.069825 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7aaaadc-f2e3-4472-b59f-6eff602952a1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:18.093597 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7aaaadc-f2e3-4472-b59f-6eff602952a1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:18.095059 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7aaaadc-f2e3-4472-b59f-6eff602952a1 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:18.125442 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-e7aaaadc-f2e3-4472-b59f-6eff602952a1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 09:05:18.125442 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:05:18.125442 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:05:18.125442 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:05:18.125442 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:05:18.125442 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:05:18.125442 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:05:18.125442 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:05:18.125442 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:05:18.125442 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:05:18.125442 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:05:18.125442 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:05:18.125442 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 09:05:18.125442 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 09:05:18.125442 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 09:05:18.126513 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 09:05:18.126513 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:05:18.126513 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:05:18.126513 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:05:18.126513 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:05:18.126513 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:05:18.126513 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:05:18.126513 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:05:18.126513 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 14 09:05:18.126513 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:05:18.126513 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:05:18.126513 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:05:18.126513 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:05:18.126513 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:05:18.126513 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:05:18.126513 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:05:18.126513 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:05:18.127439 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 09:05:18.127439 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 09:05:18.127439 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 09:05:18.127439 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 14 09:05:18.127439 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 391/790] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jul 14 09:05:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 326 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 09:05:18.136187 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7aaaadc-f2e3-4472-b59f-6eff602952a1 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:18.167990 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a975106d-64fd-4427-b9b0-52c74f7da9d5 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:18.206091 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9fbe1b-3c35-4e79-8da2-145f0fece79d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:18.206907 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9fbe1b-3c35-4e79-8da2-145f0fece79d None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:18.207330 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9fbe1b-3c35-4e79-8da2-145f0fece79d None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:18.207679 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9fbe1b-3c35-4e79-8da2-145f0fece79d None admin] PATH_INFO: `/v3/domains` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:18.209329 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9fbe1b-3c35-4e79-8da2-145f0fece79d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:05:18.209643 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9fbe1b-3c35-4e79-8da2-145f0fece79d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:18.211020 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9fbe1b-3c35-4e79-8da2-145f0fece79d None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:18.229278 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 400/791] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Jul 14 09:05:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 09:05:18.237363 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9fbe1b-3c35-4e79-8da2-145f0fece79d None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:18.259926 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:18.305343 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:18.341675 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7aaaadc-f2e3-4472-b59f-6eff602952a1 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:18.395031 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1ca81b-6070-4b15-9927-88e0ed9f809b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:18.395723 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1ca81b-6070-4b15-9927-88e0ed9f809b None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:18.396003 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1ca81b-6070-4b15-9927-88e0ed9f809b None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:18.396304 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1ca81b-6070-4b15-9927-88e0ed9f809b None admin] PATH_INFO: `/v3/domains` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:18.397033 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1ca81b-6070-4b15-9927-88e0ed9f809b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:05:18.397335 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1ca81b-6070-4b15-9927-88e0ed9f809b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:18.398700 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1ca81b-6070-4b15-9927-88e0ed9f809b None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:18.404730 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9fbe1b-3c35-4e79-8da2-145f0fece79d None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:18.426911 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 392/792] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Jul 14 09:05:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:05:18.432185 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1ca81b-6070-4b15-9927-88e0ed9f809b None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:18.478096 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125ae7d2-a388-4776-93d4-271224f97f09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:18.478912 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125ae7d2-a388-4776-93d4-271224f97f09 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:18.479217 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125ae7d2-a388-4776-93d4-271224f97f09 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:18.479492 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125ae7d2-a388-4776-93d4-271224f97f09 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:18.494400 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125ae7d2-a388-4776-93d4-271224f97f09 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:18.495879 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125ae7d2-a388-4776-93d4-271224f97f09 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:18.526161 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-125ae7d2-a388-4776-93d4-271224f97f09 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 14 09:05:18.526161 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:05:18.526161 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:05:18.526161 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:05:18.526161 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:05:18.526161 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:05:18.526161 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:05:18.526161 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:05:18.526161 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:05:18.526161 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:05:18.526161 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:05:18.526161 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:05:18.526161 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 09:05:18.526161 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 09:05:18.526161 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 09:05:18.526161 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 09:05:18.527174 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:05:18.527174 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:05:18.527174 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 09:05:18.527174 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 09:05:18.527174 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 09:05:18.527174 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 09:05:18.527174 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:05:18.527174 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:05:18.527174 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:05:18.527174 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:05:18.527174 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:05:18.527174 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 14 09:05:18.527174 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:05:18.527174 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:05:18.527174 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:05:18.527174 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:05:18.527174 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:05:18.527174 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:05:18.529410 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:05:18.529410 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:05:18.529410 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 09:05:18.529410 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 09:05:18.529410 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 09:05:18.529410 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 09:05:18.529410 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 09:05:18.529410 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 09:05:18.529410 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 14 09:05:18.529410 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 14 09:05:18.529410 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 401/793] 10.222.0.26 () {56 vars in 1096 bytes} [Thu Jul 14 09:05:18 2022] GET /identity/v3/users/swift => generated 83 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:05:18.534298 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125ae7d2-a388-4776-93d4-271224f97f09 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:18.617903 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1ca81b-6070-4b15-9927-88e0ed9f809b None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:18.669235 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:18.693864 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4663fee-57ae-4360-9fdb-1ee01dd330a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:18.694687 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4663fee-57ae-4360-9fdb-1ee01dd330a7 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:18.694964 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4663fee-57ae-4360-9fdb-1ee01dd330a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:18.695251 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4663fee-57ae-4360-9fdb-1ee01dd330a7 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:18.711546 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4663fee-57ae-4360-9fdb-1ee01dd330a7 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:18.713006 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4663fee-57ae-4360-9fdb-1ee01dd330a7 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:18.729788 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:18.737280 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125ae7d2-a388-4776-93d4-271224f97f09 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:18.737989 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-f4663fee-57ae-4360-9fdb-1ee01dd330a7 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 14 09:05:18.737989 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:05:18.737989 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:05:18.737989 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:05:18.737989 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:05:18.737989 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:05:18.737989 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:05:18.737989 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:05:18.737989 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:05:18.737989 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:05:18.737989 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:05:18.737989 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:05:18.737989 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 09:05:18.737989 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 09:05:18.737989 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 09:05:18.737989 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 09:05:18.740267 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:05:18.740267 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:05:18.740267 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 09:05:18.740267 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 09:05:18.740267 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 09:05:18.740267 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 09:05:18.740267 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:05:18.740267 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:05:18.740267 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:05:18.740267 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:05:18.740267 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:05:18.740267 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 14 09:05:18.740267 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:05:18.740267 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:05:18.740267 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:05:18.740267 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:05:18.740267 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:05:18.740267 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:05:18.742522 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:05:18.742522 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:05:18.742522 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 09:05:18.742522 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 09:05:18.742522 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 09:05:18.742522 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 09:05:18.742522 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 09:05:18.742522 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 09:05:18.742522 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 14 09:05:18.742522 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 14 09:05:18.742522 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 393/794] 10.222.0.26 () {56 vars in 1117 bytes} [Thu Jul 14 09:05:18 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 308 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 09:05:18.747298 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4663fee-57ae-4360-9fdb-1ee01dd330a7 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:18.813783 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b03c77-690a-4db2-8f14-1beed07cad8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:18.814460 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b03c77-690a-4db2-8f14-1beed07cad8a None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:18.814737 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b03c77-690a-4db2-8f14-1beed07cad8a None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:18.815005 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b03c77-690a-4db2-8f14-1beed07cad8a None admin] PATH_INFO: `/v3/users` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:18.815879 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b03c77-690a-4db2-8f14-1beed07cad8a None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:05:18.816181 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b03c77-690a-4db2-8f14-1beed07cad8a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:18.823229 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b03c77-690a-4db2-8f14-1beed07cad8a None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:18.836952 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 402/795] 10.222.0.26 () {56 vars in 1135 bytes} [Thu Jul 14 09:05:18 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:05:18.842465 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b03c77-690a-4db2-8f14-1beed07cad8a None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:18.945907 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4663fee-57ae-4360-9fdb-1ee01dd330a7 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:19.013801 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5251009e-0e2b-413f-8119-41b05e73661f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:19.014469 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5251009e-0e2b-413f-8119-41b05e73661f None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:19.014746 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5251009e-0e2b-413f-8119-41b05e73661f None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:19.015016 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5251009e-0e2b-413f-8119-41b05e73661f None admin] PATH_INFO: `/v3/users` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:19.015882 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5251009e-0e2b-413f-8119-41b05e73661f None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:05:19.016187 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5251009e-0e2b-413f-8119-41b05e73661f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:19.017622 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5251009e-0e2b-413f-8119-41b05e73661f None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:19.030096 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b03c77-690a-4db2-8f14-1beed07cad8a None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:19.058575 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 394/796] 10.222.0.26 () {56 vars in 1149 bytes} [Thu Jul 14 09:05:18 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:05:19.065348 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5251009e-0e2b-413f-8119-41b05e73661f None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:19.103299 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66863402-8b29-4026-9aeb-0e30f707719b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:19.105820 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66863402-8b29-4026-9aeb-0e30f707719b None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:19.106182 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66863402-8b29-4026-9aeb-0e30f707719b None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:19.106459 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66863402-8b29-4026-9aeb-0e30f707719b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:19.118020 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66863402-8b29-4026-9aeb-0e30f707719b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:19.119657 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66863402-8b29-4026-9aeb-0e30f707719b None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:19.120550 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:19.141370 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-66863402-8b29-4026-9aeb-0e30f707719b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 09:05:19.141370 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:05:19.141370 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:05:19.141370 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:05:19.141370 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:05:19.141370 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:05:19.141370 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:05:19.141370 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:05:19.141370 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:05:19.141370 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:05:19.141370 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:05:19.141370 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:05:19.141370 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 09:05:19.141370 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 09:05:19.141370 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 09:05:19.142362 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 09:05:19.142362 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:05:19.142362 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:05:19.142362 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:05:19.142362 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:05:19.142362 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:05:19.142362 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:05:19.142362 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:05:19.142362 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 14 09:05:19.142362 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:05:19.142362 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:05:19.142362 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:05:19.142362 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:05:19.142362 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:05:19.142362 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:05:19.142362 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:05:19.142362 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:05:19.143278 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 09:05:19.143278 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 09:05:19.143278 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 09:05:19.143278 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 14 09:05:19.143278 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 403/797] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jul 14 09:05:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:05:19.144177 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cee011-5315-481e-9fe0-cc51e7c9acc0 None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:19.144459 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cee011-5315-481e-9fe0-cc51e7c9acc0 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:19.145031 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cee011-5315-481e-9fe0-cc51e7c9acc0 None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:19.146174 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 404/798] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:05:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:05:19.150775 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cee011-5315-481e-9fe0-cc51e7c9acc0 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:19.162762 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:19.188391 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5251009e-0e2b-413f-8119-41b05e73661f None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:19.223633 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2cd33a-3ec5-4c87-a9f6-010b2a380192 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:19.224828 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2cd33a-3ec5-4c87-a9f6-010b2a380192 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:19.225302 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2cd33a-3ec5-4c87-a9f6-010b2a380192 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:19.225779 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2cd33a-3ec5-4c87-a9f6-010b2a380192 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:19.236492 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2cd33a-3ec5-4c87-a9f6-010b2a380192 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:19.238217 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2cd33a-3ec5-4c87-a9f6-010b2a380192 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:19.256386 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-3c2cd33a-3ec5-4c87-a9f6-010b2a380192 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 09:05:19.256386 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:05:19.256386 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:05:19.256386 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:05:19.256386 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:05:19.256386 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:05:19.256386 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:05:19.256386 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:05:19.256386 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:05:19.256386 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:05:19.256386 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:05:19.256386 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:05:19.256386 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 09:05:19.256386 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 09:05:19.256386 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 09:05:19.257396 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 09:05:19.257396 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:05:19.257396 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:05:19.257396 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:05:19.257396 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:05:19.257396 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:05:19.257396 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:05:19.257396 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:05:19.257396 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 14 09:05:19.257396 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:05:19.257396 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:05:19.257396 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:05:19.257396 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:05:19.257396 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:05:19.257396 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:05:19.257396 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:05:19.257396 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:05:19.258288 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 09:05:19.258288 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 09:05:19.258288 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 09:05:19.258288 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 14 09:05:19.260664 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cee011-5315-481e-9fe0-cc51e7c9acc0 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:19.260889 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 395/799] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jul 14 09:05:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:05:19.262273 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1688dbc-4316-4353-b721-ce5435427c70 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:19.262798 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1688dbc-4316-4353-b721-ce5435427c70 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:19.263308 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1688dbc-4316-4353-b721-ce5435427c70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:19.281623 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-d1688dbc-4316-4353-b721-ce5435427c70 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:05:19.283152 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-d1688dbc-4316-4353-b721-ce5435427c70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:05:19.303744 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0348128e-354c-4b3f-9ec5-123d6cdf6e79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:19.304459 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0348128e-354c-4b3f-9ec5-123d6cdf6e79 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:19.304790 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0348128e-354c-4b3f-9ec5-123d6cdf6e79 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:19.305135 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0348128e-354c-4b3f-9ec5-123d6cdf6e79 None admin] PATH_INFO: `/v3/domains` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:19.305972 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0348128e-354c-4b3f-9ec5-123d6cdf6e79 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:05:19.306503 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0348128e-354c-4b3f-9ec5-123d6cdf6e79 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:19.307999 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0348128e-354c-4b3f-9ec5-123d6cdf6e79 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:19.321344 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 405/800] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Jul 14 09:05:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:05:19.322273 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0348128e-354c-4b3f-9ec5-123d6cdf6e79 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:19.371292 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 396/801] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:05:19 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 09:05:19.372531 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1688dbc-4316-4353-b721-ce5435427c70 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:19.402067 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:19.415699 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:19.443468 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0348128e-354c-4b3f-9ec5-123d6cdf6e79 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:19.477501 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69bcc16-ff3c-4dd4-9949-68e5dabfd6d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:19.478087 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69bcc16-ff3c-4dd4-9949-68e5dabfd6d9 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:19.478392 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69bcc16-ff3c-4dd4-9949-68e5dabfd6d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:19.478627 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69bcc16-ff3c-4dd4-9949-68e5dabfd6d9 None admin] PATH_INFO: `/v3/domains` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:19.479379 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69bcc16-ff3c-4dd4-9949-68e5dabfd6d9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:05:19.479686 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69bcc16-ff3c-4dd4-9949-68e5dabfd6d9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:19.481048 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69bcc16-ff3c-4dd4-9949-68e5dabfd6d9 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:19.497408 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 406/802] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Jul 14 09:05:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:05:19.497408 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69bcc16-ff3c-4dd4-9949-68e5dabfd6d9 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:19.500990 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1688dbc-4316-4353-b721-ce5435427c70 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:19.538416 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f15d51-ecfb-4c1f-911b-4bd1b9b95086 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:19.539252 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f15d51-ecfb-4c1f-911b-4bd1b9b95086 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:19.539586 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f15d51-ecfb-4c1f-911b-4bd1b9b95086 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:19.539881 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f15d51-ecfb-4c1f-911b-4bd1b9b95086 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:19.547475 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f15d51-ecfb-4c1f-911b-4bd1b9b95086 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:19.548936 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f15d51-ecfb-4c1f-911b-4bd1b9b95086 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:19.566087 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-f5f15d51-ecfb-4c1f-911b-4bd1b9b95086 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 09:05:19.566087 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:05:19.566087 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:05:19.566087 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:05:19.566087 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:05:19.566087 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:05:19.566087 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:05:19.566087 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:05:19.566087 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:05:19.566087 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:05:19.566087 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:05:19.566087 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:05:19.566087 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 09:05:19.566087 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 09:05:19.566087 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 09:05:19.567003 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 09:05:19.567003 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:05:19.567003 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:05:19.567003 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:05:19.567003 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:05:19.567003 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:05:19.567003 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:05:19.567003 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:05:19.567003 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 14 09:05:19.567003 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:05:19.567003 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:05:19.567003 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:05:19.567003 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:05:19.567003 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:05:19.567003 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:05:19.567003 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:05:19.567003 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:05:19.568289 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 09:05:19.568289 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 09:05:19.568289 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 09:05:19.568289 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 09:05:19.568289 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 09:05:19.568289 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 09:05:19.568289 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 09:05:19.568289 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 14 09:05:19.572152 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 397/803] 10.222.0.26 () {56 vars in 1111 bytes} [Thu Jul 14 09:05:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:05:19.573436 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd7ce22-c159-4400-a4b7-50e927e7657f None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:19.573719 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd7ce22-c159-4400-a4b7-50e927e7657f None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:19.574613 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd7ce22-c159-4400-a4b7-50e927e7657f None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:19.579820 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 398/804] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:05:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:05:19.579820 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd7ce22-c159-4400-a4b7-50e927e7657f None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:19.606979 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69bcc16-ff3c-4dd4-9949-68e5dabfd6d9 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:19.634917 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:19.644130 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4df2bb6-2e8d-4951-a9aa-776317a8e9ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:19.644925 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4df2bb6-2e8d-4951-a9aa-776317a8e9ee None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:19.645203 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4df2bb6-2e8d-4951-a9aa-776317a8e9ee None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:19.645469 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4df2bb6-2e8d-4951-a9aa-776317a8e9ee None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:19.658752 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4df2bb6-2e8d-4951-a9aa-776317a8e9ee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 09:05:19.665725 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:19.667167 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-f4df2bb6-2e8d-4951-a9aa-776317a8e9ee None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 09:05:19.667167 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:05:19.667167 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:05:19.667167 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:05:19.667167 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:05:19.667167 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:05:19.667167 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:05:19.667167 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:05:19.667167 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:05:19.667167 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:05:19.667167 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:05:19.667167 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:05:19.667167 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 09:05:19.667167 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 09:05:19.667167 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 09:05:19.667167 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 09:05:19.668087 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 09:05:19.668087 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 09:05:19.668087 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 09:05:19.668087 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 09:05:19.668087 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:05:19.668087 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:05:19.668087 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:05:19.668087 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:05:19.668087 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:05:19.668087 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:05:19.668087 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:05:19.668087 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 14 09:05:19.668087 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:05:19.668087 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:05:19.668087 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:05:19.668087 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:05:19.674429 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:05:19.674429 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:05:19.674429 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:05:19.674429 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:05:19.674429 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 09:05:19.674429 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 09:05:19.674429 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 09:05:19.674429 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 09:05:19.674429 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 09:05:19.674429 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 09:05:19.674429 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 09:05:19.674429 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 14 09:05:19.674429 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 407/805] 10.222.0.26 () {56 vars in 1096 bytes} [Thu Jul 14 09:05:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:05:19.674429 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4df2bb6-2e8d-4951-a9aa-776317a8e9ee None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:19.689499 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd7ce22-c159-4400-a4b7-50e927e7657f None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:19.727113 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e9cd2a-1e16-40d9-adfb-c143c42c84ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:19.728239 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e9cd2a-1e16-40d9-adfb-c143c42c84ac None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:19.728682 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e9cd2a-1e16-40d9-adfb-c143c42c84ac None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:19.728956 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e9cd2a-1e16-40d9-adfb-c143c42c84ac None admin] PATH_INFO: `/v3/projects/service` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:19.736261 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e9cd2a-1e16-40d9-adfb-c143c42c84ac None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:19.737711 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e9cd2a-1e16-40d9-adfb-c143c42c84ac None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:19.751234 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-96e9cd2a-1e16-40d9-adfb-c143c42c84ac None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 09:05:19.751234 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:05:19.751234 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:05:19.751234 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:05:19.751234 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:05:19.751234 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:05:19.751234 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:05:19.751234 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:05:19.751234 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:05:19.751234 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:05:19.751234 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:05:19.751234 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:05:19.751234 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 09:05:19.751234 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 09:05:19.751234 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 09:05:19.752191 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 09:05:19.752191 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:05:19.752191 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:05:19.752191 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:05:19.752191 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:05:19.752191 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:05:19.752191 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:05:19.752191 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:05:19.752191 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 14 09:05:19.752191 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:05:19.752191 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:05:19.752191 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:05:19.752191 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:05:19.752191 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:05:19.752191 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:05:19.752191 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:05:19.752191 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:05:19.753098 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 09:05:19.753098 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 09:05:19.753098 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 09:05:19.753098 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 09:05:19.753098 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 09:05:19.753098 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 09:05:19.753098 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 09:05:19.753098 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 14 09:05:19.753098 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 399/806] 10.222.0.26 () {56 vars in 1111 bytes} [Thu Jul 14 09:05:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:05:19.753587 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd13a21b-185a-445d-b505-3dc2dc3096d1 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:19.754087 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd13a21b-185a-445d-b505-3dc2dc3096d1 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:19.754087 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd13a21b-185a-445d-b505-3dc2dc3096d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:19.771883 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-fd13a21b-185a-445d-b505-3dc2dc3096d1 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:05:19.773050 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-fd13a21b-185a-445d-b505-3dc2dc3096d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:05:19.777805 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4df2bb6-2e8d-4951-a9aa-776317a8e9ee None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:19.809680 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bc903a-cfc5-402d-ba7b-826b9ad38c70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:19.810331 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bc903a-cfc5-402d-ba7b-826b9ad38c70 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:19.810610 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bc903a-cfc5-402d-ba7b-826b9ad38c70 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:19.810878 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bc903a-cfc5-402d-ba7b-826b9ad38c70 None admin] PATH_INFO: `/v3/projects` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:19.811683 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bc903a-cfc5-402d-ba7b-826b9ad38c70 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:05:19.811986 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bc903a-cfc5-402d-ba7b-826b9ad38c70 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:19.813409 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bc903a-cfc5-402d-ba7b-826b9ad38c70 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:19.825118 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 408/807] 10.222.0.26 () {56 vars in 1148 bytes} [Thu Jul 14 09:05:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:05:19.825887 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bc903a-cfc5-402d-ba7b-826b9ad38c70 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:19.837876 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 400/808] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:05:19 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 09:05:19.839081 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6bff47-0b0c-4d96-8cef-c0335037785e None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:19.839398 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6bff47-0b0c-4d96-8cef-c0335037785e None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:19.839675 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6bff47-0b0c-4d96-8cef-c0335037785e None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:19.840927 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 401/809] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:05:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:05:19.841607 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6bff47-0b0c-4d96-8cef-c0335037785e None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:19.879824 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:19.921971 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:19.930503 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bc903a-cfc5-402d-ba7b-826b9ad38c70 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:19.948822 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6bff47-0b0c-4d96-8cef-c0335037785e None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:19.963130 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4dd638-0c4c-4036-a85d-cf4593c2df43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:19.963820 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4dd638-0c4c-4036-a85d-cf4593c2df43 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:19.964103 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4dd638-0c4c-4036-a85d-cf4593c2df43 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:19.964382 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4dd638-0c4c-4036-a85d-cf4593c2df43 None admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:19.965128 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4dd638-0c4c-4036-a85d-cf4593c2df43 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:05:19.965429 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4dd638-0c4c-4036-a85d-cf4593c2df43 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:19.966823 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4dd638-0c4c-4036-a85d-cf4593c2df43 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:19.975140 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 409/810] 10.222.0.26 () {56 vars in 1099 bytes} [Thu Jul 14 09:05:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:05:19.975751 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4dd638-0c4c-4036-a85d-cf4593c2df43 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:19.985106 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f3df2d-daa8-43d3-ad82-23e681ae5c3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:19.985705 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f3df2d-daa8-43d3-ad82-23e681ae5c3a None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:19.985987 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f3df2d-daa8-43d3-ad82-23e681ae5c3a None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:19.986256 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f3df2d-daa8-43d3-ad82-23e681ae5c3a None admin] PATH_INFO: `/v3/projects` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:19.987040 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f3df2d-daa8-43d3-ad82-23e681ae5c3a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:05:19.987378 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f3df2d-daa8-43d3-ad82-23e681ae5c3a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:19.988798 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f3df2d-daa8-43d3-ad82-23e681ae5c3a None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:20.000994 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 402/811] 10.222.0.26 () {56 vars in 1148 bytes} [Thu Jul 14 09:05:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:05:20.002287 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9d4ced-86c5-4a10-be58-45dea929090c None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:20.002575 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9d4ced-86c5-4a10-be58-45dea929090c None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:20.002855 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9d4ced-86c5-4a10-be58-45dea929090c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:20.018977 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-8f9d4ced-86c5-4a10-be58-45dea929090c None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:05:20.020185 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-8f9d4ced-86c5-4a10-be58-45dea929090c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:05:20.075263 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4dd638-0c4c-4036-a85d-cf4593c2df43 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:20.086686 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:20.088701 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 403/812] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:05:20 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 09:05:20.089618 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9d4ced-86c5-4a10-be58-45dea929090c None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:20.111424 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf74162-b7d0-42be-84ff-e70e4e17312a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:20.112336 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf74162-b7d0-42be-84ff-e70e4e17312a None admin] REQUEST_METHOD: `PUT` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:20.112609 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf74162-b7d0-42be-84ff-e70e4e17312a None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:20.113116 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf74162-b7d0-42be-84ff-e70e4e17312a None admin] PATH_INFO: `/v3/projects/33bf210d015847439655b9943d1b0261/users/e5a43c80514b4a13af03c7f8ebcf7a5f/roles/7b549d58d1724395b343609d895c80d0` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:20.114932 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf74162-b7d0-42be-84ff-e70e4e17312a None admin] RBAC: Authorizing `identity:create_grant(project_id=33bf210d015847439655b9943d1b0261, user_id=e5a43c80514b4a13af03c7f8ebcf7a5f, role_id=7b549d58d1724395b343609d895c80d0)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:20.116385 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf74162-b7d0-42be-84ff-e70e4e17312a None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:20.130216 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 410/813] 10.222.0.26 () {58 vars in 1439 bytes} [Thu Jul 14 09:05:19 2022] PUT /identity/v3/projects/33bf210d015847439655b9943d1b0261/users/e5a43c80514b4a13af03c7f8ebcf7a5f/roles/7b549d58d1724395b343609d895c80d0 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:05:20.130948 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf74162-b7d0-42be-84ff-e70e4e17312a None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:20.165718 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:20.209347 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9d4ced-86c5-4a10-be58-45dea929090c None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:20.246054 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5436df58-3203-43c5-9b99-331b40b7ffb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:20.246261 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf74162-b7d0-42be-84ff-e70e4e17312a None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:20.246797 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5436df58-3203-43c5-9b99-331b40b7ffb6 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:20.247082 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5436df58-3203-43c5-9b99-331b40b7ffb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:20.247391 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5436df58-3203-43c5-9b99-331b40b7ffb6 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:20.248069 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5436df58-3203-43c5-9b99-331b40b7ffb6 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:20.249493 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5436df58-3203-43c5-9b99-331b40b7ffb6 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:20.262378 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-5436df58-3203-43c5-9b99-331b40b7ffb6 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 14 09:05:20.262378 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:05:20.262378 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 09:05:20.262378 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 09:05:20.262378 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 09:05:20.262378 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 09:05:20.262378 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 09:05:20.262378 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 09:05:20.262378 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 14 09:05:20.262378 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 14 09:05:20.262378 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 09:05:20.262378 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 14 09:05:20.262378 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:05:20.262378 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:05:20.262378 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:05:20.262378 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:05:20.263462 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:05:20.263462 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:05:20.263462 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:05:20.263462 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:05:20.263462 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:05:20.263462 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:05:20.263462 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:05:20.263462 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 09:05:20.263462 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 09:05:20.263462 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 09:05:20.263462 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 09:05:20.263462 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:05:20.263462 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:05:20.263462 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:05:20.263462 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:05:20.263462 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:05:20.264321 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:05:20.264321 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:05:20.264321 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 14 09:05:20.264321 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:05:20.264321 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:05:20.264321 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:05:20.264321 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:05:20.264321 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:05:20.264321 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:05:20.264321 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:05:20.264321 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:05:20.264321 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 09:05:20.264321 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 09:05:20.264321 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 14 09:05:20.264321 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 14 09:05:20.264321 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 404/814] 10.222.0.26 () {56 vars in 1129 bytes} [Thu Jul 14 09:05:20 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:05:20.265904 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5436df58-3203-43c5-9b99-331b40b7ffb6 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:20.296150 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7d8f90-89a5-47b1-a5c3-9ed92b5d2b36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:20.296965 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7d8f90-89a5-47b1-a5c3-9ed92b5d2b36 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:20.297235 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7d8f90-89a5-47b1-a5c3-9ed92b5d2b36 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:20.297509 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7d8f90-89a5-47b1-a5c3-9ed92b5d2b36 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:20.308066 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7d8f90-89a5-47b1-a5c3-9ed92b5d2b36 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:20.309516 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7d8f90-89a5-47b1-a5c3-9ed92b5d2b36 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:20.327410 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-cc7d8f90-89a5-47b1-a5c3-9ed92b5d2b36 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 09:05:20.327410 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:05:20.327410 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:05:20.327410 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:05:20.327410 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:05:20.327410 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:05:20.327410 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:05:20.327410 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:05:20.327410 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:05:20.327410 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:05:20.327410 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:05:20.327410 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:05:20.327410 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 09:05:20.327410 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 09:05:20.327410 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 09:05:20.328338 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 09:05:20.328338 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:05:20.328338 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:05:20.328338 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:05:20.328338 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:05:20.328338 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:05:20.328338 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:05:20.328338 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:05:20.328338 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 14 09:05:20.328338 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:05:20.328338 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:05:20.328338 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:05:20.328338 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:05:20.328338 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:05:20.328338 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:05:20.328338 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:05:20.328338 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:05:20.329268 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 09:05:20.329268 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 09:05:20.329268 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 09:05:20.329268 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 14 09:05:20.329268 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 411/815] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jul 14 09:05:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:05:20.329565 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7d8f90-89a5-47b1-a5c3-9ed92b5d2b36 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:20.355699 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:20.381977 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5436df58-3203-43c5-9b99-331b40b7ffb6 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:20.408619 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:20.419438 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27766e8-b507-416a-a207-214da59f0d5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:20.420407 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27766e8-b507-416a-a207-214da59f0d5b None admin] REQUEST_METHOD: `PUT` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:20.420689 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27766e8-b507-416a-a207-214da59f0d5b None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:20.420961 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27766e8-b507-416a-a207-214da59f0d5b None admin] PATH_INFO: `/v3/projects/33bf210d015847439655b9943d1b0261/users/7d6267ede8d64e2abe1f4058a9797fb7/roles/f5c487bc605f45df82ef6367c80fae70` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:20.441901 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7d8f90-89a5-47b1-a5c3-9ed92b5d2b36 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:20.442450 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27766e8-b507-416a-a207-214da59f0d5b None admin] RBAC: Authorizing `identity:create_grant(project_id=33bf210d015847439655b9943d1b0261, user_id=7d6267ede8d64e2abe1f4058a9797fb7, role_id=f5c487bc605f45df82ef6367c80fae70)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:20.444211 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27766e8-b507-416a-a207-214da59f0d5b None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:20.460038 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 405/816] 10.222.0.26 () {58 vars in 1439 bytes} [Thu Jul 14 09:05:20 2022] PUT /identity/v3/projects/33bf210d015847439655b9943d1b0261/users/7d6267ede8d64e2abe1f4058a9797fb7/roles/f5c487bc605f45df82ef6367c80fae70 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:05:20.461200 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27766e8-b507-416a-a207-214da59f0d5b None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:20.495439 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb89aadf-e15a-426c-9dba-11fa42532304 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:20.496680 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb89aadf-e15a-426c-9dba-11fa42532304 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:20.497165 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb89aadf-e15a-426c-9dba-11fa42532304 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:20.497586 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb89aadf-e15a-426c-9dba-11fa42532304 None admin] PATH_INFO: `/v3/services/network` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:20.498425 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb89aadf-e15a-426c-9dba-11fa42532304 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:20.500158 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb89aadf-e15a-426c-9dba-11fa42532304 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:20.513916 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-fb89aadf-e15a-426c-9dba-11fa42532304 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 14 09:05:20.513916 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:05:20.513916 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 09:05:20.513916 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 09:05:20.513916 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 09:05:20.513916 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 09:05:20.513916 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 09:05:20.513916 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 09:05:20.513916 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 14 09:05:20.513916 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 14 09:05:20.513916 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 09:05:20.513916 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 14 09:05:20.513916 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:05:20.513916 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:05:20.513916 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:05:20.513916 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:05:20.513916 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:05:20.515246 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:05:20.515246 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:05:20.515246 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:05:20.515246 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:05:20.515246 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:05:20.515246 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:05:20.515246 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 09:05:20.515246 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 09:05:20.515246 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 09:05:20.515246 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 09:05:20.515246 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:05:20.515246 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:05:20.515246 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:05:20.515246 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:05:20.515246 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:05:20.515246 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:05:20.516373 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:05:20.516373 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 14 09:05:20.516373 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:05:20.516373 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:05:20.516373 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:05:20.516373 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:05:20.516373 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:05:20.516373 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:05:20.516373 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:05:20.516373 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:05:20.516373 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 09:05:20.516373 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 09:05:20.516373 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 14 09:05:20.516373 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 14 09:05:20.524222 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 412/817] 10.222.0.26 () {56 vars in 1111 bytes} [Thu Jul 14 09:05:20 2022] GET /identity/v3/services/network => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:05:20.525546 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb89aadf-e15a-426c-9dba-11fa42532304 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:20.619371 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27766e8-b507-416a-a207-214da59f0d5b None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:20.648005 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:20.670072 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a61874-25bd-403a-b206-8d986e2388b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:20.670908 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb89aadf-e15a-426c-9dba-11fa42532304 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:20.671288 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a61874-25bd-403a-b206-8d986e2388b2 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:20.671710 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a61874-25bd-403a-b206-8d986e2388b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:20.672132 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a61874-25bd-403a-b206-8d986e2388b2 None admin] PATH_INFO: `/v3/services` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:20.674066 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a61874-25bd-403a-b206-8d986e2388b2 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:05:20.674529 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a61874-25bd-403a-b206-8d986e2388b2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:20.676142 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a61874-25bd-403a-b206-8d986e2388b2 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:20.683025 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 406/818] 10.222.0.26 () {56 vars in 1124 bytes} [Thu Jul 14 09:05:20 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:05:20.684117 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a61874-25bd-403a-b206-8d986e2388b2 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:20.711957 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06707bb8-c5cc-474f-ba39-154cc487a671 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:20.712624 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06707bb8-c5cc-474f-ba39-154cc487a671 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:20.712902 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06707bb8-c5cc-474f-ba39-154cc487a671 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:20.713170 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06707bb8-c5cc-474f-ba39-154cc487a671 None admin] PATH_INFO: `/v3/domains` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:20.713891 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06707bb8-c5cc-474f-ba39-154cc487a671 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:05:20.714244 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06707bb8-c5cc-474f-ba39-154cc487a671 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:20.715874 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06707bb8-c5cc-474f-ba39-154cc487a671 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:20.729556 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 413/819] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Jul 14 09:05:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:05:20.730535 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06707bb8-c5cc-474f-ba39-154cc487a671 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:20.738036 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:20.812149 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a61874-25bd-403a-b206-8d986e2388b2 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:20.854456 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eaf9029-a05c-4631-8468-b6e78b7db62e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:20.855406 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaf9029-a05c-4631-8468-b6e78b7db62e None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:20.855683 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaf9029-a05c-4631-8468-b6e78b7db62e None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:20.855959 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaf9029-a05c-4631-8468-b6e78b7db62e None admin] PATH_INFO: `/v3/services` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:20.856917 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eaf9029-a05c-4631-8468-b6e78b7db62e None admin] RBAC: Adding query filter params (name=network) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:05:20.857224 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eaf9029-a05c-4631-8468-b6e78b7db62e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:20.859294 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eaf9029-a05c-4631-8468-b6e78b7db62e None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:20.860603 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06707bb8-c5cc-474f-ba39-154cc487a671 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:20.865740 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 407/820] 10.222.0.26 () {56 vars in 1112 bytes} [Thu Jul 14 09:05:20 2022] GET /identity/v3/services?name=network => generated 124 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:05:20.866663 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eaf9029-a05c-4631-8468-b6e78b7db62e None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:20.908003 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e011f16-83ec-48bf-a1c7-0de89fe77560 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:20.908732 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e011f16-83ec-48bf-a1c7-0de89fe77560 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:20.909013 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e011f16-83ec-48bf-a1c7-0de89fe77560 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:20.909283 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e011f16-83ec-48bf-a1c7-0de89fe77560 None admin] PATH_INFO: `/v3/services` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:20.910733 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e011f16-83ec-48bf-a1c7-0de89fe77560 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:05:20.911084 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e011f16-83ec-48bf-a1c7-0de89fe77560 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:20.913192 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e011f16-83ec-48bf-a1c7-0de89fe77560 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:20.923275 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 414/821] 10.222.0.26 () {56 vars in 1124 bytes} [Thu Jul 14 09:05:20 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:05:20.924239 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e011f16-83ec-48bf-a1c7-0de89fe77560 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:20.938428 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:20.992450 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eaf9029-a05c-4631-8468-b6e78b7db62e None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:21.001864 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:21.032528 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a275e222-5247-4d9c-a541-2a27524cc3fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:21.033269 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a275e222-5247-4d9c-a541-2a27524cc3fc None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:21.033667 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a275e222-5247-4d9c-a541-2a27524cc3fc None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:21.034034 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a275e222-5247-4d9c-a541-2a27524cc3fc None admin] PATH_INFO: `/v3/users/nova` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:21.042106 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a275e222-5247-4d9c-a541-2a27524cc3fc None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:21.043671 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a275e222-5247-4d9c-a541-2a27524cc3fc None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:21.048330 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e011f16-83ec-48bf-a1c7-0de89fe77560 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:21.059836 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-a275e222-5247-4d9c-a541-2a27524cc3fc None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 14 09:05:21.059836 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:05:21.059836 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:05:21.059836 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:05:21.059836 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:05:21.059836 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:05:21.059836 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:05:21.059836 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:05:21.059836 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:05:21.059836 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:05:21.059836 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:05:21.059836 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:05:21.059836 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 09:05:21.059836 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 09:05:21.059836 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 09:05:21.059836 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 09:05:21.061008 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:05:21.061008 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:05:21.061008 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 09:05:21.061008 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 09:05:21.061008 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 09:05:21.061008 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 09:05:21.061008 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:05:21.061008 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:05:21.061008 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:05:21.061008 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:05:21.061008 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:05:21.061008 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 14 09:05:21.061008 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:05:21.061008 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:05:21.061008 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:05:21.061008 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:05:21.061008 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:05:21.061008 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:05:21.062260 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:05:21.062260 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:05:21.062260 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 09:05:21.062260 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 09:05:21.062260 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 09:05:21.062260 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 09:05:21.062260 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 09:05:21.062260 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 09:05:21.062260 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 14 09:05:21.062260 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 14 09:05:21.064448 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 408/822] 10.222.0.26 () {56 vars in 1093 bytes} [Thu Jul 14 09:05:20 2022] GET /identity/v3/users/nova => generated 82 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:05:21.065285 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a275e222-5247-4d9c-a541-2a27524cc3fc None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:21.095402 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7fcd3b-663a-4b41-b24b-f7df1a99ee59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:21.097193 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7fcd3b-663a-4b41-b24b-f7df1a99ee59 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:21.098416 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7fcd3b-663a-4b41-b24b-f7df1a99ee59 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:21.099380 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7fcd3b-663a-4b41-b24b-f7df1a99ee59 None admin] PATH_INFO: `/v3/services` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:21.100278 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7fcd3b-663a-4b41-b24b-f7df1a99ee59 None admin] RBAC: Adding query filter params (type=network) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:05:21.100976 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7fcd3b-663a-4b41-b24b-f7df1a99ee59 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:21.104339 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7fcd3b-663a-4b41-b24b-f7df1a99ee59 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:21.118361 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 415/823] 10.222.0.26 () {56 vars in 1112 bytes} [Thu Jul 14 09:05:20 2022] GET /identity/v3/services?type=network => generated 350 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:05:21.118361 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7fcd3b-663a-4b41-b24b-f7df1a99ee59 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:21.196597 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a275e222-5247-4d9c-a541-2a27524cc3fc None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:21.199055 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:21.227203 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7fcd3b-663a-4b41-b24b-f7df1a99ee59 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:21.244199 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185065c0-1869-4f90-8a2d-2c219f952a9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:21.245413 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185065c0-1869-4f90-8a2d-2c219f952a9e None admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:21.246206 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185065c0-1869-4f90-8a2d-2c219f952a9e None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:21.246698 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185065c0-1869-4f90-8a2d-2c219f952a9e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:21.247677 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185065c0-1869-4f90-8a2d-2c219f952a9e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:21.249866 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185065c0-1869-4f90-8a2d-2c219f952a9e None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:21.263857 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c276beb5-a261-4a82-9dae-d09be4e3d139 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:21.264736 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c276beb5-a261-4a82-9dae-d09be4e3d139 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:21.264980 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c276beb5-a261-4a82-9dae-d09be4e3d139 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:21.265315 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c276beb5-a261-4a82-9dae-d09be4e3d139 None admin] PATH_INFO: `/v3/users` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:21.266232 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c276beb5-a261-4a82-9dae-d09be4e3d139 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:05:21.266617 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c276beb5-a261-4a82-9dae-d09be4e3d139 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:21.268468 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c276beb5-a261-4a82-9dae-d09be4e3d139 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:21.286774 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 409/824] 10.222.0.26 () {60 vars in 1144 bytes} [Thu Jul 14 09:05:21 2022] POST /identity/v3/endpoints => generated 346 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 09:05:21.286774 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185065c0-1869-4f90-8a2d-2c219f952a9e None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:21.304128 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 416/825] 10.222.0.26 () {56 vars in 1133 bytes} [Thu Jul 14 09:05:21 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:05:21.305183 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c276beb5-a261-4a82-9dae-d09be4e3d139 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:21.339828 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:21.493484 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c276beb5-a261-4a82-9dae-d09be4e3d139 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:21.504704 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185065c0-1869-4f90-8a2d-2c219f952a9e None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:21.545059 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:21.556126 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb65f5e-5540-4648-b510-f4fcbb697d66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:21.556126 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb65f5e-5540-4648-b510-f4fcbb697d66 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:21.556538 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb65f5e-5540-4648-b510-f4fcbb697d66 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:21.557028 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb65f5e-5540-4648-b510-f4fcbb697d66 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:21.563665 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648f0774-d46b-4d18-b725-31cda08ab5f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:21.564515 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648f0774-d46b-4d18-b725-31cda08ab5f8 None admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:21.564963 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648f0774-d46b-4d18-b725-31cda08ab5f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:21.565380 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648f0774-d46b-4d18-b725-31cda08ab5f8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:21.566270 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648f0774-d46b-4d18-b725-31cda08ab5f8 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:21.568375 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648f0774-d46b-4d18-b725-31cda08ab5f8 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:21.595432 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 410/826] 10.222.0.26 () {60 vars in 1144 bytes} [Thu Jul 14 09:05:21 2022] POST /identity/v3/endpoints => generated 327 bytes in 314 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 09:05:21.648861 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb65f5e-5540-4648-b510-f4fcbb697d66 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:21.650779 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb65f5e-5540-4648-b510-f4fcbb697d66 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:21.767153 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 417/827] 10.222.0.26 () {58 vars in 1282 bytes} [Thu Jul 14 09:05:21 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 462 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 09:05:21.773291 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648f0774-d46b-4d18-b725-31cda08ab5f8 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:21.857247 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:21.890365 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648f0774-d46b-4d18-b725-31cda08ab5f8 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:21.925007 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f54879f-7050-44e9-857f-f3bbcdd41fd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:21.925810 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f54879f-7050-44e9-857f-f3bbcdd41fd2 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:21.926093 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f54879f-7050-44e9-857f-f3bbcdd41fd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:21.926370 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f54879f-7050-44e9-857f-f3bbcdd41fd2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:21.938432 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f54879f-7050-44e9-857f-f3bbcdd41fd2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:21.939993 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f54879f-7050-44e9-857f-f3bbcdd41fd2 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:21.957616 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-4f54879f-7050-44e9-857f-f3bbcdd41fd2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 09:05:21.957616 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:05:21.957616 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:05:21.957616 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:05:21.957616 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:05:21.957616 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:05:21.957616 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:05:21.957616 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:05:21.957616 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:05:21.957616 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:05:21.957616 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:05:21.957616 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:05:21.957616 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 09:05:21.957616 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 09:05:21.957616 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 09:05:21.958586 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 09:05:21.958586 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:05:21.958586 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:05:21.958586 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:05:21.958586 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:05:21.958586 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:05:21.958586 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:05:21.958586 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:05:21.958586 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 14 09:05:21.958586 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:05:21.958586 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:05:21.958586 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:05:21.958586 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:05:21.958586 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:05:21.958586 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:05:21.958586 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:05:21.958586 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:05:21.959569 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 09:05:21.959569 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 09:05:21.959569 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 09:05:21.959569 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 14 09:05:21.959569 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 411/828] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jul 14 09:05:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:05:21.967746 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb65f5e-5540-4648-b510-f4fcbb697d66 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:22.002248 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:22.076371 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb65f5e-5540-4648-b510-f4fcbb697d66 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:22.112430 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edac95b5-fbe9-45cc-8dc7-bbe557729b87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:22.113439 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edac95b5-fbe9-45cc-8dc7-bbe557729b87 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:22.113911 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edac95b5-fbe9-45cc-8dc7-bbe557729b87 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:22.114436 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edac95b5-fbe9-45cc-8dc7-bbe557729b87 None admin] PATH_INFO: `/v3/domains` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:22.115804 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edac95b5-fbe9-45cc-8dc7-bbe557729b87 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:05:22.116341 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edac95b5-fbe9-45cc-8dc7-bbe557729b87 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:22.118413 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edac95b5-fbe9-45cc-8dc7-bbe557729b87 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:22.136099 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 418/829] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Jul 14 09:05:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:05:22.141718 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f54879f-7050-44e9-857f-f3bbcdd41fd2 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:22.244702 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413c0f75-549d-4c25-9967-2491a299acdc None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:22.245053 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413c0f75-549d-4c25-9967-2491a299acdc None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:22.245259 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413c0f75-549d-4c25-9967-2491a299acdc None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:22.246753 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 419/830] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:05:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:05:22.251883 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7402bc-a745-4ace-aea0-8fd56e00676e None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:22.252408 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7402bc-a745-4ace-aea0-8fd56e00676e None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:22.252886 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7402bc-a745-4ace-aea0-8fd56e00676e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:22.255330 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f54879f-7050-44e9-857f-f3bbcdd41fd2 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:22.270051 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-ed7402bc-a745-4ace-aea0-8fd56e00676e None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:05:22.271733 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-ed7402bc-a745-4ace-aea0-8fd56e00676e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:05:22.292646 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:22.296915 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d242329-922d-4db5-9a0b-f904d21a3e6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:22.298339 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d242329-922d-4db5-9a0b-f904d21a3e6f None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:22.298636 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d242329-922d-4db5-9a0b-f904d21a3e6f None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:22.298956 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d242329-922d-4db5-9a0b-f904d21a3e6f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:22.308755 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d242329-922d-4db5-9a0b-f904d21a3e6f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:22.310852 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d242329-922d-4db5-9a0b-f904d21a3e6f None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:22.329457 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-9d242329-922d-4db5-9a0b-f904d21a3e6f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 09:05:22.329457 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:05:22.329457 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:05:22.329457 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:05:22.329457 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:05:22.329457 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:05:22.329457 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:05:22.329457 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:05:22.329457 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:05:22.329457 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:05:22.329457 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:05:22.329457 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:05:22.329457 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 09:05:22.329457 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 09:05:22.329457 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 09:05:22.330527 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 09:05:22.330527 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:05:22.330527 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:05:22.330527 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:05:22.330527 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:05:22.330527 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:05:22.330527 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:05:22.330527 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:05:22.330527 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 14 09:05:22.330527 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:05:22.330527 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:05:22.330527 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:05:22.330527 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:05:22.330527 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:05:22.330527 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:05:22.330527 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:05:22.330527 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:05:22.331943 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 09:05:22.331943 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 09:05:22.331943 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 09:05:22.331943 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 09:05:22.331943 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 09:05:22.331943 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 09:05:22.331943 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 09:05:22.331943 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 14 09:05:22.331943 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 412/831] 10.222.0.26 () {56 vars in 1111 bytes} [Thu Jul 14 09:05:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:05:22.336618 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d242329-922d-4db5-9a0b-f904d21a3e6f None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:22.363286 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 420/832] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:05:22 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 09:05:22.381055 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7402bc-a745-4ace-aea0-8fd56e00676e None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:22.405444 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:22.455866 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d242329-922d-4db5-9a0b-f904d21a3e6f None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:22.493277 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d7fb0e-e6bd-4657-93fe-bcff1ce228ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:22.494498 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d7fb0e-e6bd-4657-93fe-bcff1ce228ce None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:22.495017 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d7fb0e-e6bd-4657-93fe-bcff1ce228ce None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:22.495584 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d7fb0e-e6bd-4657-93fe-bcff1ce228ce None admin] PATH_INFO: `/v3/projects` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:22.496711 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d7fb0e-e6bd-4657-93fe-bcff1ce228ce None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:05:22.497248 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d7fb0e-e6bd-4657-93fe-bcff1ce228ce None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:22.497355 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7402bc-a745-4ace-aea0-8fd56e00676e None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:22.499560 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d7fb0e-e6bd-4657-93fe-bcff1ce228ce None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:22.512825 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 413/833] 10.222.0.26 () {56 vars in 1148 bytes} [Thu Jul 14 09:05:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 09:05:22.518705 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d7fb0e-e6bd-4657-93fe-bcff1ce228ce None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:22.543265 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f081a36-a966-4454-a7f5-77fdc7db5835 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:22.544541 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f081a36-a966-4454-a7f5-77fdc7db5835 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:22.545046 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f081a36-a966-4454-a7f5-77fdc7db5835 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:22.545519 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f081a36-a966-4454-a7f5-77fdc7db5835 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:22.560027 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f081a36-a966-4454-a7f5-77fdc7db5835 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 09:05:22.569786 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:22.570579 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-0f081a36-a966-4454-a7f5-77fdc7db5835 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 09:05:22.570579 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:05:22.570579 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:05:22.570579 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:05:22.570579 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:05:22.570579 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:05:22.570579 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:05:22.570579 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:05:22.570579 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:05:22.570579 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:05:22.570579 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:05:22.570579 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:05:22.570579 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 09:05:22.570579 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 09:05:22.570579 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 09:05:22.570579 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 09:05:22.571533 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 09:05:22.571533 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 09:05:22.571533 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 09:05:22.571533 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 09:05:22.571533 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:05:22.571533 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:05:22.571533 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:05:22.571533 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:05:22.571533 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:05:22.571533 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:05:22.571533 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:05:22.571533 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 14 09:05:22.571533 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:05:22.571533 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:05:22.571533 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:05:22.571533 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:05:22.572408 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:05:22.572408 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:05:22.572408 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:05:22.572408 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:05:22.572408 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 09:05:22.572408 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 09:05:22.572408 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 09:05:22.572408 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 09:05:22.572408 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 09:05:22.572408 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 09:05:22.572408 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 09:05:22.572408 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 14 09:05:22.580595 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 421/834] 10.222.0.26 () {56 vars in 1096 bytes} [Thu Jul 14 09:05:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 09:05:22.582414 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f081a36-a966-4454-a7f5-77fdc7db5835 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:22.630526 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d7fb0e-e6bd-4657-93fe-bcff1ce228ce None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:22.665556 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bafe9f-e957-4926-b7cc-88fdae8d8e95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:22.666487 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bafe9f-e957-4926-b7cc-88fdae8d8e95 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:22.666762 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bafe9f-e957-4926-b7cc-88fdae8d8e95 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:22.667036 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bafe9f-e957-4926-b7cc-88fdae8d8e95 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:22.667898 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bafe9f-e957-4926-b7cc-88fdae8d8e95 None admin] RBAC: Adding query filter params (role.id=7b549d58d1724395b343609d895c80d0, scope.project.id=33bf210d015847439655b9943d1b0261, user.id=538fff3c5a2b487eb44ae7c1a1198102) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:05:22.668209 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bafe9f-e957-4926-b7cc-88fdae8d8e95 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:22.669637 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bafe9f-e957-4926-b7cc-88fdae8d8e95 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:22.683003 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 414/835] 10.222.0.26 () {56 vars in 1374 bytes} [Thu Jul 14 09:05:22 2022] GET /identity/v3/role_assignments?user.id=538fff3c5a2b487eb44ae7c1a1198102&scope.project.id=33bf210d015847439655b9943d1b0261&role.id=7b549d58d1724395b343609d895c80d0 => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:05:22.683003 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bc590a-2304-4238-b2c0-37d70aad57ac None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:22.683003 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bc590a-2304-4238-b2c0-37d70aad57ac None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:22.683003 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bc590a-2304-4238-b2c0-37d70aad57ac None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:22.683003 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 415/836] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:05:22 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:05:22.684891 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6113f95b-ee1a-4d34-b4b4-c4e9ee965d47 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:22.685208 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6113f95b-ee1a-4d34-b4b4-c4e9ee965d47 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:22.685501 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6113f95b-ee1a-4d34-b4b4-c4e9ee965d47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:22.687894 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:22.705130 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-6113f95b-ee1a-4d34-b4b4-c4e9ee965d47 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:05:22.706372 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-6113f95b-ee1a-4d34-b4b4-c4e9ee965d47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:05:22.713216 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f081a36-a966-4454-a7f5-77fdc7db5835 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:22.758193 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5879d29-b228-48ab-8b84-54f810a50dcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:22.758898 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5879d29-b228-48ab-8b84-54f810a50dcc None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:22.759177 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5879d29-b228-48ab-8b84-54f810a50dcc None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:22.759488 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5879d29-b228-48ab-8b84-54f810a50dcc None admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:22.760582 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5879d29-b228-48ab-8b84-54f810a50dcc None admin] RBAC: Adding query filter params (name=admin) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:05:22.760804 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5879d29-b228-48ab-8b84-54f810a50dcc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:22.762270 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5879d29-b228-48ab-8b84-54f810a50dcc None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:22.777257 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 422/837] 10.222.0.26 () {56 vars in 1099 bytes} [Thu Jul 14 09:05:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 09:05:22.780352 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5879d29-b228-48ab-8b84-54f810a50dcc None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:22.807649 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 416/838] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:05:22 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 09:05:22.823155 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6113f95b-ee1a-4d34-b4b4-c4e9ee965d47 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:22.866281 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:22.905732 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5879d29-b228-48ab-8b84-54f810a50dcc None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:22.947145 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6113f95b-ee1a-4d34-b4b4-c4e9ee965d47 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:22.950840 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a29b12-89dc-4cd3-a89d-4ab0c1dd6550 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:22.951682 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a29b12-89dc-4cd3-a89d-4ab0c1dd6550 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:22.951967 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a29b12-89dc-4cd3-a89d-4ab0c1dd6550 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:22.952245 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a29b12-89dc-4cd3-a89d-4ab0c1dd6550 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:22.965122 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a29b12-89dc-4cd3-a89d-4ab0c1dd6550 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:22.966589 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a29b12-89dc-4cd3-a89d-4ab0c1dd6550 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:22.969195 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:22.984913 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1c3a76-3e2e-4db2-a902-503ff844f6fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:22.985703 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1c3a76-3e2e-4db2-a902-503ff844f6fe None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:22.985980 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1c3a76-3e2e-4db2-a902-503ff844f6fe None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:22.986247 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1c3a76-3e2e-4db2-a902-503ff844f6fe None admin] PATH_INFO: `/v3/roles/service` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:22.987715 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-85a29b12-89dc-4cd3-a89d-4ab0c1dd6550 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 09:05:22.987715 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:05:22.987715 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:05:22.987715 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:05:22.987715 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:05:22.987715 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:05:22.987715 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:05:22.987715 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:05:22.987715 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:05:22.987715 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:05:22.987715 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:05:22.987715 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:05:22.987715 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 09:05:22.987715 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 09:05:22.987715 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 09:05:22.988598 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 09:05:22.988598 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:05:22.988598 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:05:22.988598 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:05:22.988598 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:05:22.988598 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:05:22.988598 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:05:22.988598 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:05:22.988598 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 14 09:05:22.988598 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:05:22.988598 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:05:22.988598 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:05:22.988598 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:05:22.988598 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:05:22.988598 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:05:22.988598 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:05:22.988598 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:05:22.989559 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 09:05:22.989559 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 09:05:22.989559 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 09:05:22.989559 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 14 09:05:22.989559 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 423/839] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jul 14 09:05:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:05:22.994826 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a29b12-89dc-4cd3-a89d-4ab0c1dd6550 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:22.999380 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1c3a76-3e2e-4db2-a902-503ff844f6fe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 09:05:23.007659 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-5e1c3a76-3e2e-4db2-a902-503ff844f6fe None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 09:05:23.007659 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:05:23.007659 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:05:23.007659 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:05:23.007659 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:05:23.007659 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:05:23.007659 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:05:23.007659 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:05:23.007659 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:05:23.007659 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:05:23.007659 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:05:23.007659 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:05:23.007659 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 09:05:23.007659 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 09:05:23.007659 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 09:05:23.007659 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 09:05:23.009773 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 09:05:23.009773 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 09:05:23.009773 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 09:05:23.009773 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 09:05:23.009773 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:05:23.009773 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:05:23.009773 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:05:23.009773 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:05:23.009773 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:05:23.009773 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:05:23.009773 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:05:23.009773 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 14 09:05:23.009773 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:05:23.009773 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:05:23.009773 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:05:23.009773 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:05:23.011832 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:05:23.011832 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:05:23.011832 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:05:23.011832 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:05:23.011832 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 09:05:23.011832 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 09:05:23.011832 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 09:05:23.011832 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 09:05:23.011832 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 09:05:23.011832 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 09:05:23.011832 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 09:05:23.011832 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 14 09:05:23.011832 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 417/840] 10.222.0.26 () {56 vars in 1102 bytes} [Thu Jul 14 09:05:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 09:05:23.019225 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1c3a76-3e2e-4db2-a902-503ff844f6fe None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:23.096891 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a29b12-89dc-4cd3-a89d-4ab0c1dd6550 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:23.118202 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:23.125953 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1c3a76-3e2e-4db2-a902-503ff844f6fe None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:23.132085 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18a96c7-864f-47a8-b356-0d849487222c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:23.132765 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18a96c7-864f-47a8-b356-0d849487222c None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:23.133111 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18a96c7-864f-47a8-b356-0d849487222c None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:23.133388 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18a96c7-864f-47a8-b356-0d849487222c None admin] PATH_INFO: `/v3/domains` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:23.134175 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18a96c7-864f-47a8-b356-0d849487222c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:05:23.134486 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18a96c7-864f-47a8-b356-0d849487222c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:23.135906 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18a96c7-864f-47a8-b356-0d849487222c None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:23.153152 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 424/841] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Jul 14 09:05:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 09:05:23.154093 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18a96c7-864f-47a8-b356-0d849487222c None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:23.162332 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec4fc49-f6ba-4bed-9718-5229d50af019 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:23.162935 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec4fc49-f6ba-4bed-9718-5229d50af019 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:23.163228 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec4fc49-f6ba-4bed-9718-5229d50af019 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:23.163499 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec4fc49-f6ba-4bed-9718-5229d50af019 None admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:23.164352 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec4fc49-f6ba-4bed-9718-5229d50af019 None admin] RBAC: Adding query filter params (name=service) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:05:23.164657 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec4fc49-f6ba-4bed-9718-5229d50af019 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:23.166184 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec4fc49-f6ba-4bed-9718-5229d50af019 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:23.180110 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 418/842] 10.222.0.26 () {56 vars in 1103 bytes} [Thu Jul 14 09:05:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:05:23.181292 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec4fc49-f6ba-4bed-9718-5229d50af019 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:23.212318 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:23.256817 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18a96c7-864f-47a8-b356-0d849487222c None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:23.293089 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4c51b5-74a2-4f51-a2c1-88bf9a4bcded None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:23.293895 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4c51b5-74a2-4f51-a2c1-88bf9a4bcded None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:23.294200 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4c51b5-74a2-4f51-a2c1-88bf9a4bcded None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:23.294479 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4c51b5-74a2-4f51-a2c1-88bf9a4bcded None admin] PATH_INFO: `/v3/users/swift` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:23.298462 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec4fc49-f6ba-4bed-9718-5229d50af019 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:23.303974 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4c51b5-74a2-4f51-a2c1-88bf9a4bcded None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:23.305615 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4c51b5-74a2-4f51-a2c1-88bf9a4bcded None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:23.321553 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-9f4c51b5-74a2-4f51-a2c1-88bf9a4bcded None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 14 09:05:23.321553 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:05:23.321553 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:05:23.321553 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:05:23.321553 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:05:23.321553 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:05:23.321553 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:05:23.321553 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:05:23.321553 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:05:23.321553 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:05:23.321553 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:05:23.321553 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:05:23.321553 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 09:05:23.321553 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 09:05:23.321553 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 09:05:23.321553 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 09:05:23.322757 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:05:23.322757 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:05:23.322757 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 09:05:23.322757 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 09:05:23.322757 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 09:05:23.322757 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 09:05:23.322757 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:05:23.322757 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:05:23.322757 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:05:23.322757 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:05:23.322757 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:05:23.322757 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 14 09:05:23.322757 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:05:23.322757 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:05:23.322757 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:05:23.322757 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:05:23.322757 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:05:23.322757 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:05:23.323957 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:05:23.323957 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:05:23.323957 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 09:05:23.323957 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 09:05:23.323957 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 09:05:23.323957 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 09:05:23.323957 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 09:05:23.323957 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 09:05:23.323957 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 14 09:05:23.323957 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 14 09:05:23.326327 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 425/843] 10.222.0.26 () {56 vars in 1096 bytes} [Thu Jul 14 09:05:23 2022] GET /identity/v3/users/swift => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:05:23.331559 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4c51b5-74a2-4f51-a2c1-88bf9a4bcded None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:23.340324 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9625f4-01ed-42ea-b3e9-358f415c22d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:23.341305 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9625f4-01ed-42ea-b3e9-358f415c22d0 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:23.342216 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9625f4-01ed-42ea-b3e9-358f415c22d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:23.342599 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9625f4-01ed-42ea-b3e9-358f415c22d0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:23.359444 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9625f4-01ed-42ea-b3e9-358f415c22d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:23.361106 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9625f4-01ed-42ea-b3e9-358f415c22d0 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:23.379135 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-6c9625f4-01ed-42ea-b3e9-358f415c22d0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 09:05:23.379135 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:05:23.379135 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:05:23.379135 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:05:23.379135 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:05:23.379135 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:05:23.379135 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:05:23.379135 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:05:23.379135 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:05:23.379135 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:05:23.379135 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:05:23.379135 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:05:23.379135 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 09:05:23.379135 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 09:05:23.379135 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 09:05:23.380161 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 09:05:23.380161 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:05:23.380161 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:05:23.380161 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:05:23.380161 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:05:23.380161 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:05:23.380161 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:05:23.380161 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:05:23.380161 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 14 09:05:23.380161 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:05:23.380161 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:05:23.380161 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:05:23.380161 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:05:23.380161 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:05:23.380161 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:05:23.380161 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:05:23.380161 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:05:23.381056 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 09:05:23.381056 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 09:05:23.381056 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 09:05:23.381056 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 14 09:05:23.387471 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 419/844] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jul 14 09:05:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 09:05:23.388907 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9625f4-01ed-42ea-b3e9-358f415c22d0 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:23.402110 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:23.451453 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4c51b5-74a2-4f51-a2c1-88bf9a4bcded None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:23.468868 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:23.495091 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4861ba2-a0c1-43fc-9f17-94032765b826 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:23.496181 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4861ba2-a0c1-43fc-9f17-94032765b826 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:23.496659 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4861ba2-a0c1-43fc-9f17-94032765b826 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:23.497125 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4861ba2-a0c1-43fc-9f17-94032765b826 None admin] PATH_INFO: `/v3/users` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:23.498169 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4861ba2-a0c1-43fc-9f17-94032765b826 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:05:23.499121 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4861ba2-a0c1-43fc-9f17-94032765b826 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:23.500893 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4861ba2-a0c1-43fc-9f17-94032765b826 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:23.523242 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9625f4-01ed-42ea-b3e9-358f415c22d0 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:23.523823 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 426/845] 10.222.0.26 () {56 vars in 1135 bytes} [Thu Jul 14 09:05:23 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 09:05:23.526295 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4861ba2-a0c1-43fc-9f17-94032765b826 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:23.572577 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b887cd7f-a365-485b-ae51-a116ebba21e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:23.573257 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b887cd7f-a365-485b-ae51-a116ebba21e7 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:23.573542 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b887cd7f-a365-485b-ae51-a116ebba21e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:23.573822 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b887cd7f-a365-485b-ae51-a116ebba21e7 None admin] PATH_INFO: `/v3/domains` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:23.574859 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b887cd7f-a365-485b-ae51-a116ebba21e7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:05:23.575177 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b887cd7f-a365-485b-ae51-a116ebba21e7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:23.576832 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b887cd7f-a365-485b-ae51-a116ebba21e7 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:23.597029 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 420/846] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Jul 14 09:05:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:05:23.598831 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b887cd7f-a365-485b-ae51-a116ebba21e7 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:23.645213 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4861ba2-a0c1-43fc-9f17-94032765b826 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:23.672205 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:23.692149 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bf1cf6-d0bd-45ae-ae57-4904f14d61fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:23.692811 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bf1cf6-d0bd-45ae-ae57-4904f14d61fe None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:23.693098 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bf1cf6-d0bd-45ae-ae57-4904f14d61fe None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:23.693376 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bf1cf6-d0bd-45ae-ae57-4904f14d61fe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:23.708755 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b887cd7f-a365-485b-ae51-a116ebba21e7 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:23.745390 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:23.747982 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9e681e-7c97-4fc2-af12-90ccb7232b4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:23.750247 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9e681e-7c97-4fc2-af12-90ccb7232b4e None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:23.750540 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9e681e-7c97-4fc2-af12-90ccb7232b4e None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:23.751061 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9e681e-7c97-4fc2-af12-90ccb7232b4e None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:23.761502 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9e681e-7c97-4fc2-af12-90ccb7232b4e None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:23.763605 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9e681e-7c97-4fc2-af12-90ccb7232b4e None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:23.772472 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bf1cf6-d0bd-45ae-ae57-4904f14d61fe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:23.774170 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bf1cf6-d0bd-45ae-ae57-4904f14d61fe None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:23.781244 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-9a9e681e-7c97-4fc2-af12-90ccb7232b4e None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 14 09:05:23.781244 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:05:23.781244 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:05:23.781244 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:05:23.781244 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:05:23.781244 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:05:23.781244 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:05:23.781244 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:05:23.781244 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:05:23.781244 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:05:23.781244 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:05:23.781244 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:05:23.781244 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 09:05:23.781244 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 09:05:23.781244 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 09:05:23.781244 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 09:05:23.782259 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:05:23.782259 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:05:23.782259 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 09:05:23.782259 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 09:05:23.782259 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 09:05:23.782259 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 09:05:23.782259 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:05:23.782259 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:05:23.782259 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:05:23.782259 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:05:23.782259 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:05:23.782259 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 14 09:05:23.782259 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:05:23.782259 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:05:23.782259 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:05:23.782259 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:05:23.782259 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:05:23.782259 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:05:23.783248 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:05:23.783248 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:05:23.783248 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 09:05:23.783248 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 09:05:23.783248 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 09:05:23.783248 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 09:05:23.783248 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 09:05:23.783248 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 09:05:23.783248 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 14 09:05:23.783248 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 14 09:05:23.785019 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 421/847] 10.222.0.26 () {56 vars in 1117 bytes} [Thu Jul 14 09:05:23 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 09:05:23.787420 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbe0f31-78a8-4328-9ef5-a1d444d22655 None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:23.787942 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbe0f31-78a8-4328-9ef5-a1d444d22655 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:23.788229 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbe0f31-78a8-4328-9ef5-a1d444d22655 None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:23.789685 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 422/848] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:05:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:05:23.795610 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbe0f31-78a8-4328-9ef5-a1d444d22655 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:23.890047 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 427/849] 10.222.0.26 () {58 vars in 1282 bytes} [Thu Jul 14 09:05:23 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 09:05:23.890047 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3537e7-d15b-4e09-be4f-fe14bd930671 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:23.890047 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3537e7-d15b-4e09-be4f-fe14bd930671 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:23.890047 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3537e7-d15b-4e09-be4f-fe14bd930671 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:23.907524 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbe0f31-78a8-4328-9ef5-a1d444d22655 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:23.910180 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-fc3537e7-d15b-4e09-be4f-fe14bd930671 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:05:23.911454 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-fc3537e7-d15b-4e09-be4f-fe14bd930671 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:05:23.940372 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:23.947486 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd3bf38-abbd-44a0-8bd4-878656cfd671 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:23.948168 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd3bf38-abbd-44a0-8bd4-878656cfd671 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:23.948450 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd3bf38-abbd-44a0-8bd4-878656cfd671 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:23.948731 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd3bf38-abbd-44a0-8bd4-878656cfd671 None admin] PATH_INFO: `/v3/users` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:23.949567 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd3bf38-abbd-44a0-8bd4-878656cfd671 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:05:23.949876 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd3bf38-abbd-44a0-8bd4-878656cfd671 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:23.951329 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd3bf38-abbd-44a0-8bd4-878656cfd671 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:23.971293 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 423/850] 10.222.0.26 () {56 vars in 1149 bytes} [Thu Jul 14 09:05:23 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:05:23.976402 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd3bf38-abbd-44a0-8bd4-878656cfd671 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:23.984430 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 428/851] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:05:23 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 09:05:23.989734 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3537e7-d15b-4e09-be4f-fe14bd930671 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:23.989734 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:24.088549 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd3bf38-abbd-44a0-8bd4-878656cfd671 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:24.089212 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3537e7-d15b-4e09-be4f-fe14bd930671 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:24.123957 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2fc258-c31a-48c3-86aa-aec89291145b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:24.124605 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2fc258-c31a-48c3-86aa-aec89291145b None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:24.124895 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2fc258-c31a-48c3-86aa-aec89291145b None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:24.125151 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2fc258-c31a-48c3-86aa-aec89291145b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:24.128123 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a311c85-55e9-4223-b874-282ee8a8f442 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:24.129137 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a311c85-55e9-4223-b874-282ee8a8f442 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:24.129424 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a311c85-55e9-4223-b874-282ee8a8f442 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:24.129711 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a311c85-55e9-4223-b874-282ee8a8f442 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:24.147150 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a311c85-55e9-4223-b874-282ee8a8f442 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:24.148966 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a311c85-55e9-4223-b874-282ee8a8f442 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:24.169554 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-8a311c85-55e9-4223-b874-282ee8a8f442 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 09:05:24.169554 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:05:24.169554 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:05:24.169554 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:05:24.169554 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:05:24.169554 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:05:24.169554 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:05:24.169554 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:05:24.169554 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:05:24.169554 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:05:24.169554 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:05:24.169554 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:05:24.169554 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 09:05:24.169554 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 09:05:24.169554 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 09:05:24.171077 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 09:05:24.171077 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:05:24.171077 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:05:24.171077 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:05:24.171077 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:05:24.171077 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:05:24.171077 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:05:24.171077 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:05:24.171077 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 14 09:05:24.171077 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:05:24.171077 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:05:24.171077 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:05:24.171077 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:05:24.171077 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:05:24.171077 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:05:24.171077 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:05:24.171077 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:05:24.172281 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 09:05:24.172281 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 09:05:24.172281 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 09:05:24.172281 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 14 09:05:24.172281 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 424/852] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jul 14 09:05:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:05:24.172281 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a311c85-55e9-4223-b874-282ee8a8f442 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:24.201651 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2fc258-c31a-48c3-86aa-aec89291145b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:24.203097 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2fc258-c31a-48c3-86aa-aec89291145b None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:24.205925 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:24.218786 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:24.281774 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a311c85-55e9-4223-b874-282ee8a8f442 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:24.311254 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 429/853] 10.222.0.26 () {58 vars in 1282 bytes} [Thu Jul 14 09:05:23 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 09:05:24.312168 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2fc258-c31a-48c3-86aa-aec89291145b None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:24.326031 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f71140-ff70-4c47-8303-2be1a3ebb560 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:24.327115 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f71140-ff70-4c47-8303-2be1a3ebb560 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:24.327438 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f71140-ff70-4c47-8303-2be1a3ebb560 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:24.327717 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f71140-ff70-4c47-8303-2be1a3ebb560 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:24.328526 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f71140-ff70-4c47-8303-2be1a3ebb560 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:24.330344 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f71140-ff70-4c47-8303-2be1a3ebb560 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:24.342624 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-d4f71140-ff70-4c47-8303-2be1a3ebb560 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 14 09:05:24.342624 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:05:24.342624 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 09:05:24.342624 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 09:05:24.342624 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 09:05:24.342624 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 09:05:24.342624 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 09:05:24.342624 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 09:05:24.342624 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 14 09:05:24.342624 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 14 09:05:24.342624 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 09:05:24.342624 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 14 09:05:24.342624 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:05:24.342624 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:05:24.342624 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:05:24.342624 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:05:24.342624 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:05:24.343642 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:05:24.343642 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:05:24.343642 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:05:24.343642 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:05:24.343642 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:05:24.343642 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:05:24.343642 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 09:05:24.343642 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 09:05:24.343642 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 09:05:24.343642 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 09:05:24.343642 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:05:24.343642 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:05:24.343642 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:05:24.343642 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:05:24.343642 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:05:24.343642 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:05:24.345194 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:05:24.345194 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 14 09:05:24.345194 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:05:24.345194 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:05:24.345194 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:05:24.345194 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:05:24.345194 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:05:24.345194 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:05:24.345194 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:05:24.345194 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:05:24.345194 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 09:05:24.345194 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 09:05:24.345194 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 14 09:05:24.345194 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 14 09:05:24.345194 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 425/854] 10.222.0.26 () {56 vars in 1114 bytes} [Thu Jul 14 09:05:24 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:05:24.346000 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f71140-ff70-4c47-8303-2be1a3ebb560 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:24.421579 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2fc258-c31a-48c3-86aa-aec89291145b None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:24.424165 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:24.446643 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f71140-ff70-4c47-8303-2be1a3ebb560 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:24.448985 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:24.459846 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1098837-7f2e-4237-ad49-4bbbbb26aec4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:24.460523 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1098837-7f2e-4237-ad49-4bbbbb26aec4 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:24.460805 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1098837-7f2e-4237-ad49-4bbbbb26aec4 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:24.461079 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1098837-7f2e-4237-ad49-4bbbbb26aec4 None admin] PATH_INFO: `/v3/domains` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:24.461809 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1098837-7f2e-4237-ad49-4bbbbb26aec4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:05:24.462114 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1098837-7f2e-4237-ad49-4bbbbb26aec4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:24.463504 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1098837-7f2e-4237-ad49-4bbbbb26aec4 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:24.474999 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 430/855] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Jul 14 09:05:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:05:24.475831 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1098837-7f2e-4237-ad49-4bbbbb26aec4 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:24.487175 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8089349-5a86-43c5-9ade-fac64f8bcbcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:24.488008 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8089349-5a86-43c5-9ade-fac64f8bcbcc None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:24.488326 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8089349-5a86-43c5-9ade-fac64f8bcbcc None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:24.488639 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8089349-5a86-43c5-9ade-fac64f8bcbcc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:24.499892 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8089349-5a86-43c5-9ade-fac64f8bcbcc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:24.501365 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8089349-5a86-43c5-9ade-fac64f8bcbcc None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:24.522977 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-b8089349-5a86-43c5-9ade-fac64f8bcbcc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 09:05:24.522977 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:05:24.522977 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:05:24.522977 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:05:24.522977 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:05:24.522977 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:05:24.522977 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:05:24.522977 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:05:24.522977 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:05:24.522977 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:05:24.522977 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:05:24.522977 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:05:24.522977 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 09:05:24.522977 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 09:05:24.522977 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 09:05:24.524144 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 09:05:24.524144 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:05:24.524144 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:05:24.524144 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:05:24.524144 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:05:24.524144 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:05:24.524144 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:05:24.524144 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:05:24.524144 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 14 09:05:24.524144 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:05:24.524144 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:05:24.524144 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:05:24.524144 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:05:24.524144 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:05:24.524144 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:05:24.524144 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:05:24.524144 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:05:24.525233 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 09:05:24.525233 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 09:05:24.525233 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 09:05:24.525233 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 14 09:05:24.527085 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 426/856] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jul 14 09:05:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:05:24.528024 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8089349-5a86-43c5-9ade-fac64f8bcbcc None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:24.583500 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1098837-7f2e-4237-ad49-4bbbbb26aec4 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:24.618729 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219263d7-7b59-4e4f-b30e-4656a85b04d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:24.619676 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219263d7-7b59-4e4f-b30e-4656a85b04d6 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:24.619961 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219263d7-7b59-4e4f-b30e-4656a85b04d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:24.620241 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219263d7-7b59-4e4f-b30e-4656a85b04d6 None admin] PATH_INFO: `/v3/services` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:24.620951 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219263d7-7b59-4e4f-b30e-4656a85b04d6 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:05:24.621254 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219263d7-7b59-4e4f-b30e-4656a85b04d6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:24.622629 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219263d7-7b59-4e4f-b30e-4656a85b04d6 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:24.631800 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 431/857] 10.222.0.26 () {56 vars in 1114 bytes} [Thu Jul 14 09:05:24 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:05:24.631800 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219263d7-7b59-4e4f-b30e-4656a85b04d6 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:24.649997 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8089349-5a86-43c5-9ade-fac64f8bcbcc None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:24.658190 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:24.684688 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fa5c92-866d-4c36-8a07-fb46fd71dd02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:24.685484 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fa5c92-866d-4c36-8a07-fb46fd71dd02 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:24.686177 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fa5c92-866d-4c36-8a07-fb46fd71dd02 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:24.686458 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fa5c92-866d-4c36-8a07-fb46fd71dd02 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:24.694493 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fa5c92-866d-4c36-8a07-fb46fd71dd02 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:24.695962 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fa5c92-866d-4c36-8a07-fb46fd71dd02 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:24.710846 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-36fa5c92-866d-4c36-8a07-fb46fd71dd02 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 09:05:24.710846 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:05:24.710846 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:05:24.710846 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:05:24.710846 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:05:24.710846 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:05:24.710846 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:05:24.710846 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:05:24.710846 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:05:24.710846 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:05:24.710846 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:05:24.710846 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:05:24.710846 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 09:05:24.710846 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 09:05:24.710846 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 09:05:24.711860 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 09:05:24.711860 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:05:24.711860 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:05:24.711860 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:05:24.711860 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:05:24.711860 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:05:24.711860 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:05:24.711860 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:05:24.711860 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 14 09:05:24.711860 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:05:24.711860 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:05:24.711860 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:05:24.711860 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:05:24.711860 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:05:24.711860 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:05:24.711860 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:05:24.711860 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:05:24.717610 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 09:05:24.717610 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 09:05:24.717610 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 09:05:24.717610 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 09:05:24.717610 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 09:05:24.717610 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 09:05:24.717610 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 09:05:24.717610 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 14 09:05:24.717610 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 427/858] 10.222.0.26 () {56 vars in 1111 bytes} [Thu Jul 14 09:05:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:05:24.717610 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fa5c92-866d-4c36-8a07-fb46fd71dd02 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:24.735737 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:24.737330 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219263d7-7b59-4e4f-b30e-4656a85b04d6 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:24.776200 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0a0273-f5ec-4eef-8697-af51a8cdbfc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:24.776859 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0a0273-f5ec-4eef-8697-af51a8cdbfc7 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:24.777158 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0a0273-f5ec-4eef-8697-af51a8cdbfc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:24.777419 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0a0273-f5ec-4eef-8697-af51a8cdbfc7 None admin] PATH_INFO: `/v3/domains` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:24.778160 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0a0273-f5ec-4eef-8697-af51a8cdbfc7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:05:24.778465 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0a0273-f5ec-4eef-8697-af51a8cdbfc7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:24.779856 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0a0273-f5ec-4eef-8697-af51a8cdbfc7 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:24.793235 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 432/859] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Jul 14 09:05:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:05:24.794050 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0a0273-f5ec-4eef-8697-af51a8cdbfc7 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:24.824655 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fa5c92-866d-4c36-8a07-fb46fd71dd02 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:24.858041 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27f4fd7-d308-49c0-897e-3937f628fcf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:24.858718 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27f4fd7-d308-49c0-897e-3937f628fcf0 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:24.859006 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27f4fd7-d308-49c0-897e-3937f628fcf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:24.859314 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27f4fd7-d308-49c0-897e-3937f628fcf0 None admin] PATH_INFO: `/v3/services` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:24.860028 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27f4fd7-d308-49c0-897e-3937f628fcf0 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:05:24.860335 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27f4fd7-d308-49c0-897e-3937f628fcf0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:24.861709 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27f4fd7-d308-49c0-897e-3937f628fcf0 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:24.866242 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 428/860] 10.222.0.26 () {56 vars in 1114 bytes} [Thu Jul 14 09:05:24 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:05:24.867010 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27f4fd7-d308-49c0-897e-3937f628fcf0 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:24.890221 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:24.904689 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0a0273-f5ec-4eef-8697-af51a8cdbfc7 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:24.944343 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d4184f-b353-44d0-ba26-47dcbe926f51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:24.945512 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d4184f-b353-44d0-ba26-47dcbe926f51 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:24.945512 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d4184f-b353-44d0-ba26-47dcbe926f51 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:24.945718 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d4184f-b353-44d0-ba26-47dcbe926f51 None admin] PATH_INFO: `/v3/projects` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:24.946346 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d4184f-b353-44d0-ba26-47dcbe926f51 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:05:24.946653 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d4184f-b353-44d0-ba26-47dcbe926f51 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:24.948158 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d4184f-b353-44d0-ba26-47dcbe926f51 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:24.957018 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:24.961984 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 433/861] 10.222.0.26 () {56 vars in 1148 bytes} [Thu Jul 14 09:05:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:05:24.963165 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6279f52a-351a-4b73-8a71-6528c211531d None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:24.963503 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6279f52a-351a-4b73-8a71-6528c211531d None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:24.963779 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6279f52a-351a-4b73-8a71-6528c211531d None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:24.965179 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 434/862] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:05:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:05:24.967916 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6279f52a-351a-4b73-8a71-6528c211531d None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:24.988600 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27f4fd7-d308-49c0-897e-3937f628fcf0 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:25.028274 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea7132c-c270-4939-87bd-869127415a9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:25.029076 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea7132c-c270-4939-87bd-869127415a9f None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:25.029367 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea7132c-c270-4939-87bd-869127415a9f None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:25.029634 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea7132c-c270-4939-87bd-869127415a9f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:25.037642 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea7132c-c270-4939-87bd-869127415a9f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:25.038594 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea7132c-c270-4939-87bd-869127415a9f None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:25.055802 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-aea7132c-c270-4939-87bd-869127415a9f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 09:05:25.055802 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:05:25.055802 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:05:25.055802 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:05:25.055802 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:05:25.055802 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:05:25.055802 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:05:25.055802 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:05:25.055802 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:05:25.055802 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:05:25.055802 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:05:25.055802 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:05:25.055802 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 09:05:25.055802 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 09:05:25.055802 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 09:05:25.056948 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 09:05:25.056948 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:05:25.056948 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:05:25.056948 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:05:25.056948 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:05:25.056948 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:05:25.056948 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:05:25.056948 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:05:25.056948 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 14 09:05:25.056948 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:05:25.056948 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:05:25.056948 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:05:25.056948 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:05:25.056948 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:05:25.056948 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:05:25.056948 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:05:25.056948 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:05:25.057919 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 09:05:25.057919 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 09:05:25.057919 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 09:05:25.057919 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 09:05:25.057919 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 09:05:25.057919 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 09:05:25.057919 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 09:05:25.057919 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 14 09:05:25.057919 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 429/863] 10.222.0.26 () {56 vars in 1111 bytes} [Thu Jul 14 09:05:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:05:25.058434 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e323dfec-e824-4552-b35a-3140eab9bf99 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:25.058503 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e323dfec-e824-4552-b35a-3140eab9bf99 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:25.059339 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e323dfec-e824-4552-b35a-3140eab9bf99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:25.076339 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-e323dfec-e824-4552-b35a-3140eab9bf99 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:05:25.077423 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-e323dfec-e824-4552-b35a-3140eab9bf99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:05:25.087664 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6279f52a-351a-4b73-8a71-6528c211531d None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:25.123973 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ef9a4d-8e18-4167-9c32-b6cd2073c013 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:25.124633 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ef9a4d-8e18-4167-9c32-b6cd2073c013 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:25.124915 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ef9a4d-8e18-4167-9c32-b6cd2073c013 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:25.125187 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ef9a4d-8e18-4167-9c32-b6cd2073c013 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:25.126024 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ef9a4d-8e18-4167-9c32-b6cd2073c013 None admin] RBAC: Adding query filter params (role.id=7b549d58d1724395b343609d895c80d0, scope.project.id=33bf210d015847439655b9943d1b0261, user.id=e5a43c80514b4a13af03c7f8ebcf7a5f) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:05:25.126334 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ef9a4d-8e18-4167-9c32-b6cd2073c013 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:25.127803 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ef9a4d-8e18-4167-9c32-b6cd2073c013 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:25.133902 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 435/864] 10.222.0.26 () {56 vars in 1374 bytes} [Thu Jul 14 09:05:24 2022] GET /identity/v3/role_assignments?user.id=e5a43c80514b4a13af03c7f8ebcf7a5f&scope.project.id=33bf210d015847439655b9943d1b0261&role.id=7b549d58d1724395b343609d895c80d0 => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 09:05:25.134678 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ef9a4d-8e18-4167-9c32-b6cd2073c013 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:25.160094 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 430/865] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:05:25 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 09:05:25.164653 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:25.183747 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e323dfec-e824-4552-b35a-3140eab9bf99 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:25.262518 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ef9a4d-8e18-4167-9c32-b6cd2073c013 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:25.267503 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:25.302344 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad0bb9f-aecb-443d-a3d7-fc3c1216dc8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:25.303253 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad0bb9f-aecb-443d-a3d7-fc3c1216dc8c None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:25.303650 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad0bb9f-aecb-443d-a3d7-fc3c1216dc8c None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:25.304021 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad0bb9f-aecb-443d-a3d7-fc3c1216dc8c None admin] PATH_INFO: `/v3/projects` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:25.304934 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad0bb9f-aecb-443d-a3d7-fc3c1216dc8c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:05:25.305343 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad0bb9f-aecb-443d-a3d7-fc3c1216dc8c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:25.306963 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad0bb9f-aecb-443d-a3d7-fc3c1216dc8c None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:25.309149 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e323dfec-e824-4552-b35a-3140eab9bf99 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:25.327047 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 436/866] 10.222.0.26 () {56 vars in 1148 bytes} [Thu Jul 14 09:05:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:05:25.329029 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad0bb9f-aecb-443d-a3d7-fc3c1216dc8c None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:25.366547 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a439d0bd-ab75-48ed-a4ef-62389416f5a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:25.368739 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a439d0bd-ab75-48ed-a4ef-62389416f5a4 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:25.368739 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a439d0bd-ab75-48ed-a4ef-62389416f5a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:25.368739 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a439d0bd-ab75-48ed-a4ef-62389416f5a4 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:25.368739 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a439d0bd-ab75-48ed-a4ef-62389416f5a4 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:25.371233 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a439d0bd-ab75-48ed-a4ef-62389416f5a4 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:25.383154 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-a439d0bd-ab75-48ed-a4ef-62389416f5a4 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 14 09:05:25.383154 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:05:25.383154 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 09:05:25.383154 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 09:05:25.383154 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 09:05:25.383154 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 09:05:25.383154 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 09:05:25.383154 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 09:05:25.383154 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 14 09:05:25.383154 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 14 09:05:25.383154 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 09:05:25.383154 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 14 09:05:25.383154 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:05:25.383154 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:05:25.383154 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:05:25.383154 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:05:25.384261 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:05:25.384261 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:05:25.384261 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:05:25.384261 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:05:25.384261 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:05:25.384261 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:05:25.384261 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:05:25.384261 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 09:05:25.384261 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 09:05:25.384261 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 09:05:25.384261 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 09:05:25.384261 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:05:25.384261 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:05:25.384261 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:05:25.384261 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:05:25.384261 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:05:25.385115 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:05:25.385115 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:05:25.385115 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 14 09:05:25.385115 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:05:25.385115 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:05:25.385115 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:05:25.385115 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:05:25.385115 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:05:25.385115 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:05:25.385115 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:05:25.385115 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:05:25.385115 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 09:05:25.385115 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 09:05:25.385115 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 14 09:05:25.385115 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 14 09:05:25.385115 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 431/867] 10.222.0.26 () {56 vars in 1132 bytes} [Thu Jul 14 09:05:25 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 09:05:25.391223 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a439d0bd-ab75-48ed-a4ef-62389416f5a4 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:25.423491 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:25.464818 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad0bb9f-aecb-443d-a3d7-fc3c1216dc8c None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:25.512331 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3710619-0aec-48b0-8f3a-2ab6fbf1a35e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:25.513310 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3710619-0aec-48b0-8f3a-2ab6fbf1a35e None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:25.513776 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3710619-0aec-48b0-8f3a-2ab6fbf1a35e None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:25.514232 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3710619-0aec-48b0-8f3a-2ab6fbf1a35e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:25.514367 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a439d0bd-ab75-48ed-a4ef-62389416f5a4 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:25.515407 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3710619-0aec-48b0-8f3a-2ab6fbf1a35e None admin] RBAC: Adding query filter params (role.id=f5c487bc605f45df82ef6367c80fae70, scope.project.id=33bf210d015847439655b9943d1b0261, user.id=7d6267ede8d64e2abe1f4058a9797fb7) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:05:25.515936 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3710619-0aec-48b0-8f3a-2ab6fbf1a35e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:25.518717 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3710619-0aec-48b0-8f3a-2ab6fbf1a35e None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:25.527293 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 437/868] 10.222.0.26 () {56 vars in 1374 bytes} [Thu Jul 14 09:05:25 2022] GET /identity/v3/role_assignments?user.id=7d6267ede8d64e2abe1f4058a9797fb7&scope.project.id=33bf210d015847439655b9943d1b0261&role.id=f5c487bc605f45df82ef6367c80fae70 => generated 608 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 09:05:25.564603 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381d1bc4-c8a9-4e41-a894-a9c496fedabe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:25.565294 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381d1bc4-c8a9-4e41-a894-a9c496fedabe None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:25.565582 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381d1bc4-c8a9-4e41-a894-a9c496fedabe None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:25.565862 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381d1bc4-c8a9-4e41-a894-a9c496fedabe None admin] PATH_INFO: `/v3/services` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:25.566644 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381d1bc4-c8a9-4e41-a894-a9c496fedabe None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:05:25.566954 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381d1bc4-c8a9-4e41-a894-a9c496fedabe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:25.568496 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381d1bc4-c8a9-4e41-a894-a9c496fedabe None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:25.569821 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:25.575537 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 432/869] 10.222.0.26 () {56 vars in 1126 bytes} [Thu Jul 14 09:05:25 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:05:25.580823 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3710619-0aec-48b0-8f3a-2ab6fbf1a35e None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:25.697697 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3710619-0aec-48b0-8f3a-2ab6fbf1a35e None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:25.732884 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d6a42e-c6c4-4588-95f5-3c90730adce7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:25.733566 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d6a42e-c6c4-4588-95f5-3c90730adce7 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:25.733853 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d6a42e-c6c4-4588-95f5-3c90730adce7 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:25.734128 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d6a42e-c6c4-4588-95f5-3c90730adce7 None admin] PATH_INFO: `/v3/services` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:25.734831 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d6a42e-c6c4-4588-95f5-3c90730adce7 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:05:25.735132 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d6a42e-c6c4-4588-95f5-3c90730adce7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:25.736614 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d6a42e-c6c4-4588-95f5-3c90730adce7 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:25.739384 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:25.743602 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 438/870] 10.222.0.26 () {56 vars in 1126 bytes} [Thu Jul 14 09:05:25 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 09:05:26.960936 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfbb45a-8921-4c41-ac40-938160aa4f6a None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:26.961480 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfbb45a-8921-4c41-ac40-938160aa4f6a None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:26.961480 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfbb45a-8921-4c41-ac40-938160aa4f6a None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:26.963003 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 433/871] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:05:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:05:26.968343 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f717360d-b07f-4a5b-ac1b-de8813a9629e None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:26.968758 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f717360d-b07f-4a5b-ac1b-de8813a9629e None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:26.968945 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f717360d-b07f-4a5b-ac1b-de8813a9629e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:26.985917 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-f717360d-b07f-4a5b-ac1b-de8813a9629e None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:05:26.987079 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-f717360d-b07f-4a5b-ac1b-de8813a9629e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:05:27.060413 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 439/872] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:05:26 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 09:05:27.082343 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfbb45a-8921-4c41-ac40-938160aa4f6a None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:27.191243 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfbb45a-8921-4c41-ac40-938160aa4f6a None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:27.227078 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5737fe9-c036-4719-844f-ac9350d3dd9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:27.227786 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5737fe9-c036-4719-844f-ac9350d3dd9f None admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:27.228072 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5737fe9-c036-4719-844f-ac9350d3dd9f None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:27.228346 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5737fe9-c036-4719-844f-ac9350d3dd9f None admin] PATH_INFO: `/v3/services` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:27.229088 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5737fe9-c036-4719-844f-ac9350d3dd9f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:27.230460 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5737fe9-c036-4719-844f-ac9350d3dd9f None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:27.246734 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 434/873] 10.222.0.26 () {60 vars in 1141 bytes} [Thu Jul 14 09:05:27 2022] POST /identity/v3/services => generated 251 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 09:05:27.378087 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809b799f-4567-49d3-876e-988c38ff0b99 None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:27.378965 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809b799f-4567-49d3-876e-988c38ff0b99 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:27.379492 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809b799f-4567-49d3-876e-988c38ff0b99 None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:27.381445 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 440/874] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:05:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:05:27.387345 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ace676e-ae84-48b1-822b-07670ea4e30b None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:27.387953 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ace676e-ae84-48b1-822b-07670ea4e30b None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:27.388457 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ace676e-ae84-48b1-822b-07670ea4e30b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:27.406203 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-0ace676e-ae84-48b1-822b-07670ea4e30b None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:05:27.407742 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-0ace676e-ae84-48b1-822b-07670ea4e30b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:05:27.464696 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:27.486155 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 435/875] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:05:27 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 09:05:27.508242 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809b799f-4567-49d3-876e-988c38ff0b99 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:27.624407 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809b799f-4567-49d3-876e-988c38ff0b99 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:27.654245 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:27.661990 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b43e6e-70e1-47c1-94e7-cc6745866e07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:27.662818 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b43e6e-70e1-47c1-94e7-cc6745866e07 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:27.663099 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b43e6e-70e1-47c1-94e7-cc6745866e07 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:27.663399 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b43e6e-70e1-47c1-94e7-cc6745866e07 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:27.664063 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b43e6e-70e1-47c1-94e7-cc6745866e07 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:27.665438 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b43e6e-70e1-47c1-94e7-cc6745866e07 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:27.677943 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-37b43e6e-70e1-47c1-94e7-cc6745866e07 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 14 09:05:27.677943 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:05:27.677943 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 09:05:27.677943 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 09:05:27.677943 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 09:05:27.677943 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 09:05:27.677943 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 09:05:27.677943 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 09:05:27.677943 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 14 09:05:27.677943 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 14 09:05:27.677943 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 09:05:27.677943 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 14 09:05:27.677943 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:05:27.677943 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:05:27.677943 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:05:27.677943 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:05:27.677943 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:05:27.678975 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:05:27.678975 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:05:27.678975 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:05:27.678975 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:05:27.678975 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:05:27.678975 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:05:27.678975 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 09:05:27.678975 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 09:05:27.678975 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 09:05:27.678975 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 09:05:27.678975 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:05:27.678975 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:05:27.678975 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:05:27.678975 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:05:27.678975 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:05:27.678975 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:05:27.680532 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:05:27.680532 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 14 09:05:27.680532 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:05:27.680532 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:05:27.680532 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:05:27.680532 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:05:27.680532 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:05:27.680532 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:05:27.680532 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:05:27.680532 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:05:27.680532 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 09:05:27.680532 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 09:05:27.680532 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 14 09:05:27.680532 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 14 09:05:27.680532 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 441/876] 10.222.0.26 () {56 vars in 1126 bytes} [Thu Jul 14 09:05:27 2022] GET /identity/v3/services/object-store => generated 93 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:05:27.685060 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ace676e-ae84-48b1-822b-07670ea4e30b None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:27.706002 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6e3ef0-c7d9-4a9a-a4ac-158437727a30 None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:27.706428 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6e3ef0-c7d9-4a9a-a4ac-158437727a30 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:27.706737 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6e3ef0-c7d9-4a9a-a4ac-158437727a30 None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:27.708390 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 442/877] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:05:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 09:05:27.713261 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b9b749-39de-42cc-98a0-d24f80c82a63 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:27.713555 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b9b749-39de-42cc-98a0-d24f80c82a63 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:27.713831 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b9b749-39de-42cc-98a0-d24f80c82a63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:27.734626 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-c2b9b749-39de-42cc-98a0-d24f80c82a63 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:05:27.736274 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-c2b9b749-39de-42cc-98a0-d24f80c82a63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:05:27.790205 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ace676e-ae84-48b1-822b-07670ea4e30b None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:27.820859 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 443/878] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:05:27 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 09:05:27.822188 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc4f0ff-1735-43b8-9fb4-51e2128545a2 None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:27.822474 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc4f0ff-1735-43b8-9fb4-51e2128545a2 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:27.822743 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc4f0ff-1735-43b8-9fb4-51e2128545a2 None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:27.826567 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadd66bd-a312-4c5f-abc8-e5757f596cca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:27.827598 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadd66bd-a312-4c5f-abc8-e5757f596cca None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:27.828046 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 444/879] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:05:27 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:05:27.828344 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadd66bd-a312-4c5f-abc8-e5757f596cca None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:27.828806 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadd66bd-a312-4c5f-abc8-e5757f596cca None admin] PATH_INFO: `/v3/services` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:27.829192 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c95f34-0e95-4527-a371-e8222236aad7 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:27.829472 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c95f34-0e95-4527-a371-e8222236aad7 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:27.829745 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c95f34-0e95-4527-a371-e8222236aad7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:27.830339 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadd66bd-a312-4c5f-abc8-e5757f596cca None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:05:27.830832 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadd66bd-a312-4c5f-abc8-e5757f596cca None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:27.832715 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadd66bd-a312-4c5f-abc8-e5757f596cca None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:27.838788 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 436/880] 10.222.0.26 () {56 vars in 1122 bytes} [Thu Jul 14 09:05:27 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:05:27.846225 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadd66bd-a312-4c5f-abc8-e5757f596cca None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:27.847231 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-41c95f34-0e95-4527-a371-e8222236aad7 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:05:27.848365 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-41c95f34-0e95-4527-a371-e8222236aad7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:05:27.896796 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:27.919014 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 445/881] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:05:27 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 09:05:27.924104 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c95f34-0e95-4527-a371-e8222236aad7 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:27.956021 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadd66bd-a312-4c5f-abc8-e5757f596cca None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:27.986556 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:27.994839 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbddfaa-e8b2-44b6-9d07-392e3d73d22b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:27.995525 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbddfaa-e8b2-44b6-9d07-392e3d73d22b None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:27.995821 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbddfaa-e8b2-44b6-9d07-392e3d73d22b None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:27.996096 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbddfaa-e8b2-44b6-9d07-392e3d73d22b None admin] PATH_INFO: `/v3/services` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:27.996802 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbddfaa-e8b2-44b6-9d07-392e3d73d22b None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:05:27.997102 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbddfaa-e8b2-44b6-9d07-392e3d73d22b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:27.998461 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbddfaa-e8b2-44b6-9d07-392e3d73d22b None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:28.003162 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 437/882] 10.222.0.26 () {56 vars in 1122 bytes} [Thu Jul 14 09:05:27 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:05:28.004210 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbddfaa-e8b2-44b6-9d07-392e3d73d22b None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:28.055148 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c95f34-0e95-4527-a371-e8222236aad7 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:28.096945 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962a8769-0257-4fdb-ad87-934ceccdb548 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:28.097812 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962a8769-0257-4fdb-ad87-934ceccdb548 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:28.098152 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962a8769-0257-4fdb-ad87-934ceccdb548 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:28.098425 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962a8769-0257-4fdb-ad87-934ceccdb548 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:28.110634 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962a8769-0257-4fdb-ad87-934ceccdb548 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 09:05:28.119231 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-962a8769-0257-4fdb-ad87-934ceccdb548 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 14 09:05:28.119231 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:05:28.119231 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:05:28.119231 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:05:28.119231 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:05:28.119231 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:05:28.119231 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:05:28.119231 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:05:28.119231 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:05:28.119231 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:05:28.119231 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:05:28.119231 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:05:28.119231 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 09:05:28.119231 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 09:05:28.119231 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 09:05:28.119231 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 09:05:28.120292 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 09:05:28.120292 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 09:05:28.120292 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 09:05:28.120292 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 09:05:28.120292 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:05:28.120292 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:05:28.120292 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:05:28.120292 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:05:28.120292 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:05:28.120292 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:05:28.120292 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:05:28.120292 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 14 09:05:28.120292 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:05:28.120292 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:05:28.120292 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:05:28.120292 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:05:28.121162 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:05:28.121162 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:05:28.121162 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:05:28.121162 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:05:28.121162 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 09:05:28.121162 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 09:05:28.121162 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 09:05:28.121162 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 09:05:28.121162 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 09:05:28.121162 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 09:05:28.121162 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 14 09:05:28.121162 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 14 09:05:28.125777 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 446/883] 10.222.0.26 () {56 vars in 1120 bytes} [Thu Jul 14 09:05:27 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:05:28.126747 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962a8769-0257-4fdb-ad87-934ceccdb548 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:28.135763 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbddfaa-e8b2-44b6-9d07-392e3d73d22b None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:28.150871 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:28.187310 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d85aac-e4d9-45e1-b42f-864955e6093d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:28.188436 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d85aac-e4d9-45e1-b42f-864955e6093d None admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:28.188725 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d85aac-e4d9-45e1-b42f-864955e6093d None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:28.189001 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d85aac-e4d9-45e1-b42f-864955e6093d None admin] PATH_INFO: `/v3/services` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:28.189993 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d85aac-e4d9-45e1-b42f-864955e6093d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:28.191720 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d85aac-e4d9-45e1-b42f-864955e6093d None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:28.210995 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 438/884] 10.222.0.26 () {60 vars in 1141 bytes} [Thu Jul 14 09:05:28 2022] POST /identity/v3/services => generated 269 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 09:05:28.265818 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962a8769-0257-4fdb-ad87-934ceccdb548 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:28.282803 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:28.313832 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e54588-928f-4ea0-94de-3f4661fa982f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:28.314916 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e54588-928f-4ea0-94de-3f4661fa982f None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:28.315404 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e54588-928f-4ea0-94de-3f4661fa982f None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:28.315685 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e54588-928f-4ea0-94de-3f4661fa982f None admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:28.316496 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e54588-928f-4ea0-94de-3f4661fa982f None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:05:28.316798 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e54588-928f-4ea0-94de-3f4661fa982f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:28.318274 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e54588-928f-4ea0-94de-3f4661fa982f None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:28.328010 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 447/885] 10.222.0.26 () {56 vars in 1115 bytes} [Thu Jul 14 09:05:28 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:05:28.333466 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d85aac-e4d9-45e1-b42f-864955e6093d None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:28.449605 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d85aac-e4d9-45e1-b42f-864955e6093d None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:28.485109 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a034e4-e8a0-4824-9a34-b807d09b5d58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:28.486256 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a034e4-e8a0-4824-9a34-b807d09b5d58 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:28.486708 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a034e4-e8a0-4824-9a34-b807d09b5d58 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:28.487173 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a034e4-e8a0-4824-9a34-b807d09b5d58 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:28.498593 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a034e4-e8a0-4824-9a34-b807d09b5d58 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:28.500323 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a034e4-e8a0-4824-9a34-b807d09b5d58 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:28.517836 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-85a034e4-e8a0-4824-9a34-b807d09b5d58 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 09:05:28.517836 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:05:28.517836 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:05:28.517836 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:05:28.517836 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:05:28.517836 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:05:28.517836 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:05:28.517836 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:05:28.517836 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:05:28.517836 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:05:28.517836 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:05:28.517836 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:05:28.517836 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 09:05:28.517836 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 09:05:28.517836 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 09:05:28.518831 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 09:05:28.518831 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:05:28.518831 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:05:28.518831 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:05:28.518831 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:05:28.518831 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:05:28.518831 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:05:28.518831 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:05:28.518831 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 14 09:05:28.518831 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:05:28.518831 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:05:28.518831 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:05:28.518831 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:05:28.518831 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:05:28.518831 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:05:28.518831 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:05:28.518831 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:05:28.519769 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 09:05:28.519769 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 09:05:28.519769 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 09:05:28.519769 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 14 09:05:28.521773 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 439/886] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jul 14 09:05:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 09:05:28.527171 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e54588-928f-4ea0-94de-3f4661fa982f None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:28.627861 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e54588-928f-4ea0-94de-3f4661fa982f None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:28.661694 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc8edb1-7c75-4f25-ac59-9777bcf6d704 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:28.662359 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc8edb1-7c75-4f25-ac59-9777bcf6d704 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:28.662636 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc8edb1-7c75-4f25-ac59-9777bcf6d704 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:28.662906 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc8edb1-7c75-4f25-ac59-9777bcf6d704 None admin] PATH_INFO: `/v3/domains` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:28.663658 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc8edb1-7c75-4f25-ac59-9777bcf6d704 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:05:28.663955 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc8edb1-7c75-4f25-ac59-9777bcf6d704 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:28.665314 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc8edb1-7c75-4f25-ac59-9777bcf6d704 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:28.677706 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 448/887] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Jul 14 09:05:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:05:28.683060 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a034e4-e8a0-4824-9a34-b807d09b5d58 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:28.712753 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:28.790712 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a034e4-e8a0-4824-9a34-b807d09b5d58 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:28.824590 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107f7cfb-a112-4227-b7d9-0344093f0984 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:28.825402 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107f7cfb-a112-4227-b7d9-0344093f0984 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:28.825680 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107f7cfb-a112-4227-b7d9-0344093f0984 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:28.825952 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107f7cfb-a112-4227-b7d9-0344093f0984 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:28.833839 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107f7cfb-a112-4227-b7d9-0344093f0984 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:28.835372 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107f7cfb-a112-4227-b7d9-0344093f0984 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:28.849987 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-107f7cfb-a112-4227-b7d9-0344093f0984 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 14 09:05:28.849987 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:05:28.849987 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:05:28.849987 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:05:28.849987 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:05:28.849987 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:05:28.849987 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:05:28.849987 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:05:28.849987 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:05:28.849987 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:05:28.849987 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:05:28.849987 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:05:28.849987 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 09:05:28.849987 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 09:05:28.849987 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 09:05:28.849987 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 09:05:28.850915 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:05:28.850915 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:05:28.850915 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 09:05:28.850915 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 09:05:28.850915 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 09:05:28.850915 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 09:05:28.850915 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:05:28.850915 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:05:28.850915 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:05:28.850915 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:05:28.850915 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:05:28.850915 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 14 09:05:28.850915 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:05:28.850915 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:05:28.850915 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:05:28.850915 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:05:28.850915 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:05:28.850915 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:05:28.852858 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:05:28.852858 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:05:28.852858 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 09:05:28.852858 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 09:05:28.852858 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 09:05:28.852858 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 09:05:28.852858 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 09:05:28.852858 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 09:05:28.852858 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 14 09:05:28.852858 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 14 09:05:28.852858 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 440/888] 10.222.0.26 () {56 vars in 1117 bytes} [Thu Jul 14 09:05:28 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:05:28.860543 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc8edb1-7c75-4f25-ac59-9777bcf6d704 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:28.892680 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:28.961826 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc8edb1-7c75-4f25-ac59-9777bcf6d704 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:28.995398 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ac9eef-a565-43b6-a290-e990cb1af3c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:28.996172 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ac9eef-a565-43b6-a290-e990cb1af3c7 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:28.996328 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ac9eef-a565-43b6-a290-e990cb1af3c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:28.996604 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ac9eef-a565-43b6-a290-e990cb1af3c7 None admin] PATH_INFO: `/v3/users` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:28.997451 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ac9eef-a565-43b6-a290-e990cb1af3c7 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:05:28.997819 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ac9eef-a565-43b6-a290-e990cb1af3c7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:28.999360 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ac9eef-a565-43b6-a290-e990cb1af3c7 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:29.017540 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 449/889] 10.222.0.26 () {56 vars in 1149 bytes} [Thu Jul 14 09:05:28 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:05:29.024695 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107f7cfb-a112-4227-b7d9-0344093f0984 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:29.110129 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:29.127174 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107f7cfb-a112-4227-b7d9-0344093f0984 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:29.172286 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8e49dc-419c-40ab-8d34-f3e55373d7b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:29.172941 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8e49dc-419c-40ab-8d34-f3e55373d7b7 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:29.173224 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8e49dc-419c-40ab-8d34-f3e55373d7b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:29.173499 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8e49dc-419c-40ab-8d34-f3e55373d7b7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:29.244627 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8e49dc-419c-40ab-8d34-f3e55373d7b7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:29.246085 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8e49dc-419c-40ab-8d34-f3e55373d7b7 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:29.331049 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:29.333701 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c05550-2da8-4ec3-a11e-5b97989e217b None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:29.334150 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c05550-2da8-4ec3-a11e-5b97989e217b None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:29.334588 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c05550-2da8-4ec3-a11e-5b97989e217b None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:29.336341 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 450/890] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:05:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 09:05:29.341629 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05417786-4fa4-4f78-9173-9c4d6feb27a9 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:29.341909 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05417786-4fa4-4f78-9173-9c4d6feb27a9 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:29.342182 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05417786-4fa4-4f78-9173-9c4d6feb27a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:29.362859 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-05417786-4fa4-4f78-9173-9c4d6feb27a9 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:05:29.366092 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-05417786-4fa4-4f78-9173-9c4d6feb27a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:05:29.366207 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 441/891] 10.222.0.26 () {58 vars in 1282 bytes} [Thu Jul 14 09:05:29 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 09:05:29.370098 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8e49dc-419c-40ab-8d34-f3e55373d7b7 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:29.457717 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 451/892] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:05:29 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 09:05:29.475435 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05417786-4fa4-4f78-9173-9c4d6feb27a9 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:29.484258 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8e49dc-419c-40ab-8d34-f3e55373d7b7 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:29.529023 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fac0be9-d6a4-4ac1-b311-71bbe0807a2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:29.530335 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fac0be9-d6a4-4ac1-b311-71bbe0807a2f None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:29.530829 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fac0be9-d6a4-4ac1-b311-71bbe0807a2f None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:29.531382 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fac0be9-d6a4-4ac1-b311-71bbe0807a2f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:29.535787 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:29.544903 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fac0be9-d6a4-4ac1-b311-71bbe0807a2f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:29.547058 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fac0be9-d6a4-4ac1-b311-71bbe0807a2f None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:29.568847 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-8fac0be9-d6a4-4ac1-b311-71bbe0807a2f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 09:05:29.568847 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:05:29.568847 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:05:29.568847 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:05:29.568847 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:05:29.568847 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:05:29.568847 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:05:29.568847 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:05:29.568847 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:05:29.568847 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:05:29.568847 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:05:29.568847 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:05:29.568847 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 09:05:29.568847 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 09:05:29.568847 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 09:05:29.569814 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 09:05:29.569814 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:05:29.569814 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:05:29.569814 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:05:29.569814 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:05:29.569814 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:05:29.569814 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:05:29.569814 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:05:29.569814 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 14 09:05:29.569814 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:05:29.569814 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:05:29.569814 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:05:29.569814 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:05:29.569814 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:05:29.569814 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:05:29.569814 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:05:29.569814 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:05:29.570746 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 09:05:29.570746 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 09:05:29.570746 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 09:05:29.570746 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 14 09:05:29.571566 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 442/893] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jul 14 09:05:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 09:05:29.579458 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fac0be9-d6a4-4ac1-b311-71bbe0807a2f None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:29.602063 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05417786-4fa4-4f78-9173-9c4d6feb27a9 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:29.612852 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:29.647169 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a13c2f-4f40-4926-9bd4-7c4b717b2b16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:29.648483 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a13c2f-4f40-4926-9bd4-7c4b717b2b16 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:29.648908 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a13c2f-4f40-4926-9bd4-7c4b717b2b16 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:29.649319 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a13c2f-4f40-4926-9bd4-7c4b717b2b16 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:29.650374 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a13c2f-4f40-4926-9bd4-7c4b717b2b16 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:29.652479 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a13c2f-4f40-4926-9bd4-7c4b717b2b16 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:29.670496 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-72a13c2f-4f40-4926-9bd4-7c4b717b2b16 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 14 09:05:29.670496 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:05:29.670496 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 09:05:29.670496 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 09:05:29.670496 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 09:05:29.670496 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 09:05:29.670496 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 09:05:29.670496 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 09:05:29.670496 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 14 09:05:29.670496 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 14 09:05:29.670496 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 09:05:29.670496 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 14 09:05:29.670496 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:05:29.670496 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:05:29.670496 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:05:29.670496 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:05:29.670496 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:05:29.671631 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:05:29.671631 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:05:29.671631 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:05:29.671631 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:05:29.671631 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:05:29.671631 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:05:29.671631 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 09:05:29.671631 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 09:05:29.671631 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 09:05:29.671631 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 09:05:29.671631 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:05:29.671631 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:05:29.671631 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:05:29.671631 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:05:29.671631 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:05:29.671631 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:05:29.672482 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:05:29.672482 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 14 09:05:29.672482 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:05:29.672482 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:05:29.672482 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:05:29.672482 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:05:29.672482 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:05:29.672482 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:05:29.672482 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:05:29.672482 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:05:29.672482 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 09:05:29.672482 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 09:05:29.672482 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 14 09:05:29.672482 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 14 09:05:29.676115 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 452/894] 10.222.0.26 () {56 vars in 1114 bytes} [Thu Jul 14 09:05:29 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 09:05:29.681689 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a13c2f-4f40-4926-9bd4-7c4b717b2b16 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:29.690580 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fac0be9-d6a4-4ac1-b311-71bbe0807a2f None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:29.734905 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb9d92a-8dc5-4740-b619-f23a4cc8d9db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:29.736465 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb9d92a-8dc5-4740-b619-f23a4cc8d9db None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:29.737237 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb9d92a-8dc5-4740-b619-f23a4cc8d9db None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:29.737709 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb9d92a-8dc5-4740-b619-f23a4cc8d9db None admin] PATH_INFO: `/v3/domains` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:29.739264 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb9d92a-8dc5-4740-b619-f23a4cc8d9db None admin] RBAC: Adding query filter params (name=Default) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:05:29.739921 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb9d92a-8dc5-4740-b619-f23a4cc8d9db None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:29.741829 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb9d92a-8dc5-4740-b619-f23a4cc8d9db None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:29.760737 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 443/895] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Jul 14 09:05:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 09:05:29.762086 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb9d92a-8dc5-4740-b619-f23a4cc8d9db None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:29.777548 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:29.799763 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a13c2f-4f40-4926-9bd4-7c4b717b2b16 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:29.844400 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b49e29-42f3-4505-bf96-df0252a1a1c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:29.845356 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b49e29-42f3-4505-bf96-df0252a1a1c0 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:29.845777 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b49e29-42f3-4505-bf96-df0252a1a1c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:29.846219 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b49e29-42f3-4505-bf96-df0252a1a1c0 None admin] PATH_INFO: `/v3/services` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:29.847070 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b49e29-42f3-4505-bf96-df0252a1a1c0 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:05:29.847547 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b49e29-42f3-4505-bf96-df0252a1a1c0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:29.849078 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b49e29-42f3-4505-bf96-df0252a1a1c0 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:29.856453 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 453/896] 10.222.0.26 () {56 vars in 1114 bytes} [Thu Jul 14 09:05:29 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 09:05:29.863062 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b49e29-42f3-4505-bf96-df0252a1a1c0 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:29.875385 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:29.882191 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb9d92a-8dc5-4740-b619-f23a4cc8d9db None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:29.921296 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55be986f-e2e6-4a34-9ce8-db825fc1ea3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:29.922249 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55be986f-e2e6-4a34-9ce8-db825fc1ea3c None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:29.922595 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55be986f-e2e6-4a34-9ce8-db825fc1ea3c None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:29.922932 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55be986f-e2e6-4a34-9ce8-db825fc1ea3c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:29.930870 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55be986f-e2e6-4a34-9ce8-db825fc1ea3c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:29.932536 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55be986f-e2e6-4a34-9ce8-db825fc1ea3c None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:29.950652 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-55be986f-e2e6-4a34-9ce8-db825fc1ea3c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 09:05:29.950652 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:05:29.950652 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:05:29.950652 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:05:29.950652 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:05:29.950652 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:05:29.950652 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:05:29.950652 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:05:29.950652 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:05:29.950652 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:05:29.950652 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:05:29.950652 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:05:29.950652 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 09:05:29.950652 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 09:05:29.950652 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 09:05:29.951614 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 09:05:29.951614 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:05:29.951614 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:05:29.951614 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:05:29.951614 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:05:29.951614 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:05:29.951614 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:05:29.951614 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:05:29.951614 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 14 09:05:29.951614 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:05:29.951614 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:05:29.951614 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:05:29.951614 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:05:29.951614 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:05:29.951614 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:05:29.951614 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:05:29.951614 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:05:29.952526 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 09:05:29.952526 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 09:05:29.952526 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 09:05:29.952526 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 09:05:29.952526 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 09:05:29.952526 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 09:05:29.952526 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 09:05:29.952526 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 14 09:05:29.954260 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 444/897] 10.222.0.26 () {56 vars in 1111 bytes} [Thu Jul 14 09:05:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:05:29.963207 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55be986f-e2e6-4a34-9ce8-db825fc1ea3c None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:29.987980 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b49e29-42f3-4505-bf96-df0252a1a1c0 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:30.025844 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e8a161-deec-4837-99d0-6158d1ffd875 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:30.027357 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e8a161-deec-4837-99d0-6158d1ffd875 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:30.027854 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e8a161-deec-4837-99d0-6158d1ffd875 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:30.028813 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e8a161-deec-4837-99d0-6158d1ffd875 None admin] PATH_INFO: `/v3/services` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:30.029939 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e8a161-deec-4837-99d0-6158d1ffd875 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:05:30.030523 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e8a161-deec-4837-99d0-6158d1ffd875 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:30.032760 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e8a161-deec-4837-99d0-6158d1ffd875 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:30.040784 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 454/898] 10.222.0.26 () {56 vars in 1114 bytes} [Thu Jul 14 09:05:29 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:05:30.047884 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e8a161-deec-4837-99d0-6158d1ffd875 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:30.066909 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:30.090371 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55be986f-e2e6-4a34-9ce8-db825fc1ea3c None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:30.136911 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4983e395-95bf-48f9-80b5-b7679c727992 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:30.137742 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4983e395-95bf-48f9-80b5-b7679c727992 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:30.138115 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4983e395-95bf-48f9-80b5-b7679c727992 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:30.138491 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4983e395-95bf-48f9-80b5-b7679c727992 None admin] PATH_INFO: `/v3/projects` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:30.139396 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4983e395-95bf-48f9-80b5-b7679c727992 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:05:30.139798 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4983e395-95bf-48f9-80b5-b7679c727992 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:30.141317 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4983e395-95bf-48f9-80b5-b7679c727992 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:30.161761 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 445/899] 10.222.0.26 () {56 vars in 1148 bytes} [Thu Jul 14 09:05:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 09:05:30.163642 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4983e395-95bf-48f9-80b5-b7679c727992 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:30.170902 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e8a161-deec-4837-99d0-6158d1ffd875 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:30.182664 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:30.207750 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1213148f-86ac-44b5-a785-c449517694c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:30.208138 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1213148f-86ac-44b5-a785-c449517694c4 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:30.208416 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1213148f-86ac-44b5-a785-c449517694c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:30.208688 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1213148f-86ac-44b5-a785-c449517694c4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:30.209488 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1213148f-86ac-44b5-a785-c449517694c4 None admin] RBAC: Adding query filter params (interface=public, service_id=9d2555f1cc8f446e93232d6f6c847675, region_id=RegionOne) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:05:30.209797 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1213148f-86ac-44b5-a785-c449517694c4 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:30.211170 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1213148f-86ac-44b5-a785-c449517694c4 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:30.221512 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 455/900] 10.222.0.26 () {56 vars in 1251 bytes} [Thu Jul 14 09:05:30 2022] GET /identity/v3/endpoints?service_id=9d2555f1cc8f446e93232d6f6c847675&interface=public®ion_id=RegionOne => generated 194 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 09:05:30.221512 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e601de35-773f-43ee-807d-5cbf84d63f24 None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:30.221512 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e601de35-773f-43ee-807d-5cbf84d63f24 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:30.221512 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e601de35-773f-43ee-807d-5cbf84d63f24 None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:30.221512 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 456/901] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:05:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:05:30.224153 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e601de35-773f-43ee-807d-5cbf84d63f24 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:30.273545 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4983e395-95bf-48f9-80b5-b7679c727992 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:30.311243 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:30.311577 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c20ac2b-8615-4271-bf11-d618418c13f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:30.312212 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c20ac2b-8615-4271-bf11-d618418c13f1 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:30.312498 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c20ac2b-8615-4271-bf11-d618418c13f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:30.312804 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c20ac2b-8615-4271-bf11-d618418c13f1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:30.313642 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c20ac2b-8615-4271-bf11-d618418c13f1 None admin] RBAC: Adding query filter params (role.id=70488f05098d4ce09e2fb03aaaea4566, scope.project.id=33bf210d015847439655b9943d1b0261, user.id=7d6267ede8d64e2abe1f4058a9797fb7) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:05:30.313968 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c20ac2b-8615-4271-bf11-d618418c13f1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:30.315486 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c20ac2b-8615-4271-bf11-d618418c13f1 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:30.329381 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 446/902] 10.222.0.26 () {56 vars in 1374 bytes} [Thu Jul 14 09:05:30 2022] GET /identity/v3/role_assignments?user.id=7d6267ede8d64e2abe1f4058a9797fb7&scope.project.id=33bf210d015847439655b9943d1b0261&role.id=70488f05098d4ce09e2fb03aaaea4566 => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:05:30.329381 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012adedd-911c-4928-86ae-825c377832b7 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:30.329381 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012adedd-911c-4928-86ae-825c377832b7 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:30.329381 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012adedd-911c-4928-86ae-825c377832b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:30.337217 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e601de35-773f-43ee-807d-5cbf84d63f24 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:30.350591 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-012adedd-911c-4928-86ae-825c377832b7 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:05:30.352001 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-012adedd-911c-4928-86ae-825c377832b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:05:30.374599 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54960ea8-3e5e-4293-a813-9212cd9f206d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:30.375309 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54960ea8-3e5e-4293-a813-9212cd9f206d None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:30.375588 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54960ea8-3e5e-4293-a813-9212cd9f206d None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:30.375864 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54960ea8-3e5e-4293-a813-9212cd9f206d None admin] PATH_INFO: `/v3/services` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:30.376565 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54960ea8-3e5e-4293-a813-9212cd9f206d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:30.378110 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54960ea8-3e5e-4293-a813-9212cd9f206d None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:30.384723 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 457/903] 10.222.0.26 () {56 vars in 1087 bytes} [Thu Jul 14 09:05:30 2022] GET /identity/v3/services => generated 1268 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 09:05:30.386447 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6185d253-303e-4d2d-a886-335c4f87ab37 None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:30.386736 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6185d253-303e-4d2d-a886-335c4f87ab37 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:30.387016 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6185d253-303e-4d2d-a886-335c4f87ab37 None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:30.388645 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 458/904] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:05:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:05:30.394470 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24222219-b3d7-40e7-80fa-df4e73dca562 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:30.394843 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24222219-b3d7-40e7-80fa-df4e73dca562 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:30.395033 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24222219-b3d7-40e7-80fa-df4e73dca562 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:30.413666 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-24222219-b3d7-40e7-80fa-df4e73dca562 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:05:30.414958 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-24222219-b3d7-40e7-80fa-df4e73dca562 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:05:30.456738 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 447/905] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:05:30 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 09:05:30.480422 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012adedd-911c-4928-86ae-825c377832b7 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:30.496019 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:30.507257 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 459/906] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:05:30 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 09:05:30.530155 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24222219-b3d7-40e7-80fa-df4e73dca562 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:30.610651 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012adedd-911c-4928-86ae-825c377832b7 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:30.622180 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:30.653542 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1e8a2a-08ba-4ea6-9911-c99c3a255b55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:30.654585 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1e8a2a-08ba-4ea6-9911-c99c3a255b55 None admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:30.655110 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1e8a2a-08ba-4ea6-9911-c99c3a255b55 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:30.655425 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1e8a2a-08ba-4ea6-9911-c99c3a255b55 None admin] PATH_INFO: `/v3/services` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:30.656155 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1e8a2a-08ba-4ea6-9911-c99c3a255b55 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:30.657074 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24222219-b3d7-40e7-80fa-df4e73dca562 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:30.657526 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1e8a2a-08ba-4ea6-9911-c99c3a255b55 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:30.677942 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 448/907] 10.222.0.26 () {60 vars in 1141 bytes} [Thu Jul 14 09:05:30 2022] POST /identity/v3/services => generated 241 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 09:05:30.706358 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff896e07-7ce6-4f56-bbfd-066b1bf43480 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:30.707351 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff896e07-7ce6-4f56-bbfd-066b1bf43480 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:30.707624 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff896e07-7ce6-4f56-bbfd-066b1bf43480 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:30.707950 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff896e07-7ce6-4f56-bbfd-066b1bf43480 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:30.708623 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff896e07-7ce6-4f56-bbfd-066b1bf43480 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:30.710048 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff896e07-7ce6-4f56-bbfd-066b1bf43480 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:30.724057 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-ff896e07-7ce6-4f56-bbfd-066b1bf43480 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 14 09:05:30.724057 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:05:30.724057 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 09:05:30.724057 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 09:05:30.724057 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 09:05:30.724057 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 09:05:30.724057 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 09:05:30.724057 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 09:05:30.724057 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 14 09:05:30.724057 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 14 09:05:30.724057 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 09:05:30.724057 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 14 09:05:30.724057 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:05:30.724057 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:05:30.724057 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:05:30.724057 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:05:30.725071 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:05:30.725071 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:05:30.725071 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:05:30.725071 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:05:30.725071 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:05:30.725071 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:05:30.725071 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:05:30.725071 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 09:05:30.725071 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 09:05:30.725071 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 09:05:30.725071 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 09:05:30.725071 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:05:30.725071 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:05:30.725071 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:05:30.725071 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:05:30.725071 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:05:30.725939 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:05:30.725939 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:05:30.725939 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 14 09:05:30.725939 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:05:30.725939 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:05:30.725939 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:05:30.725939 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:05:30.725939 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:05:30.725939 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:05:30.725939 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:05:30.725939 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:05:30.725939 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 09:05:30.725939 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 09:05:30.725939 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 14 09:05:30.725939 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 14 09:05:30.725939 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 460/908] 10.222.0.26 () {56 vars in 1132 bytes} [Thu Jul 14 09:05:30 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 09:05:30.734018 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1e8a2a-08ba-4ea6-9911-c99c3a255b55 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:30.812610 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:30.852182 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1e8a2a-08ba-4ea6-9911-c99c3a255b55 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:30.888588 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831c0e1f-0a3a-4fdf-acd6-ebafb8bc742a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:30.889260 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831c0e1f-0a3a-4fdf-acd6-ebafb8bc742a None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:30.889651 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831c0e1f-0a3a-4fdf-acd6-ebafb8bc742a None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:30.890091 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831c0e1f-0a3a-4fdf-acd6-ebafb8bc742a None admin] PATH_INFO: `/v3/services` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:30.891150 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831c0e1f-0a3a-4fdf-acd6-ebafb8bc742a None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:05:30.891739 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831c0e1f-0a3a-4fdf-acd6-ebafb8bc742a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:30.893305 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831c0e1f-0a3a-4fdf-acd6-ebafb8bc742a None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:30.899163 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 449/909] 10.222.0.26 () {56 vars in 1126 bytes} [Thu Jul 14 09:05:30 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:05:30.905123 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff896e07-7ce6-4f56-bbfd-066b1bf43480 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:31.010655 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff896e07-7ce6-4f56-bbfd-066b1bf43480 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:31.040587 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:31.048385 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73697171-976b-450a-955e-d087d86898ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:31.049388 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73697171-976b-450a-955e-d087d86898ee None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:31.049872 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73697171-976b-450a-955e-d087d86898ee None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:31.050355 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73697171-976b-450a-955e-d087d86898ee None admin] PATH_INFO: `/v3/services` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:31.051270 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73697171-976b-450a-955e-d087d86898ee None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:05:31.051752 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73697171-976b-450a-955e-d087d86898ee None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:31.053336 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73697171-976b-450a-955e-d087d86898ee None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:31.059212 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 461/910] 10.222.0.26 () {56 vars in 1126 bytes} [Thu Jul 14 09:05:30 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:05:31.065275 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831c0e1f-0a3a-4fdf-acd6-ebafb8bc742a None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:31.170045 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831c0e1f-0a3a-4fdf-acd6-ebafb8bc742a None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:31.205603 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082153cb-825b-457c-9e9c-10a977e13553 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:31.206567 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082153cb-825b-457c-9e9c-10a977e13553 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:31.207053 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082153cb-825b-457c-9e9c-10a977e13553 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:31.207524 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082153cb-825b-457c-9e9c-10a977e13553 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:31.208492 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082153cb-825b-457c-9e9c-10a977e13553 None admin] RBAC: Adding query filter params (interface=public, service_id=fa1f201707d149c2a95f3a6449b57657, region_id=RegionOne) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:05:31.208980 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082153cb-825b-457c-9e9c-10a977e13553 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:31.210518 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082153cb-825b-457c-9e9c-10a977e13553 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:31.212821 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:31.219968 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 450/911] 10.222.0.26 () {56 vars in 1251 bytes} [Thu Jul 14 09:05:31 2022] GET /identity/v3/endpoints?service_id=fa1f201707d149c2a95f3a6449b57657&interface=public®ion_id=RegionOne => generated 194 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:05:31.225284 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73697171-976b-450a-955e-d087d86898ee None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:31.326861 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73697171-976b-450a-955e-d087d86898ee None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:31.360712 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207481d1-c214-41e6-bb61-6d83a8104bf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:31.361378 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207481d1-c214-41e6-bb61-6d83a8104bf4 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:31.361817 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207481d1-c214-41e6-bb61-6d83a8104bf4 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:31.362090 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207481d1-c214-41e6-bb61-6d83a8104bf4 None admin] PATH_INFO: `/v3/services` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:31.362771 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207481d1-c214-41e6-bb61-6d83a8104bf4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:31.364172 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207481d1-c214-41e6-bb61-6d83a8104bf4 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:31.370404 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 462/912] 10.222.0.26 () {56 vars in 1087 bytes} [Thu Jul 14 09:05:31 2022] GET /identity/v3/services => generated 1497 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 09:05:32.398098 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1017179c-404f-4acf-8434-1e64b1b4ec82 None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:32.399124 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1017179c-404f-4acf-8434-1e64b1b4ec82 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:32.399646 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1017179c-404f-4acf-8434-1e64b1b4ec82 None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:32.401519 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 451/913] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:05:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:05:32.407720 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd18d1a-772a-4065-9626-4f8f0e21e2b9 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:32.408617 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd18d1a-772a-4065-9626-4f8f0e21e2b9 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:32.409104 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd18d1a-772a-4065-9626-4f8f0e21e2b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:32.426222 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-3fd18d1a-772a-4065-9626-4f8f0e21e2b9 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:05:32.427749 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-3fd18d1a-772a-4065-9626-4f8f0e21e2b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:05:32.483079 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:32.497997 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 463/914] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:05:32 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 09:05:32.505232 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f389daf-9c49-4031-9878-1407a6637ede None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:32.505520 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f389daf-9c49-4031-9878-1407a6637ede None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:32.505799 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f389daf-9c49-4031-9878-1407a6637ede None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:32.507452 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 452/915] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:05:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:05:32.512368 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c0ce08-caad-4aa4-bcbe-249cf2e8fe21 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:32.512657 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c0ce08-caad-4aa4-bcbe-249cf2e8fe21 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:32.512931 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c0ce08-caad-4aa4-bcbe-249cf2e8fe21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:32.522471 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f389daf-9c49-4031-9878-1407a6637ede None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:32.529644 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-05c0ce08-caad-4aa4-bcbe-249cf2e8fe21 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:05:32.530822 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-05c0ce08-caad-4aa4-bcbe-249cf2e8fe21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:05:32.597795 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 464/916] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:05:32 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 09:05:32.619834 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c0ce08-caad-4aa4-bcbe-249cf2e8fe21 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:32.642028 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f389daf-9c49-4031-9878-1407a6637ede None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:32.679217 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b242fb7-c759-45c3-9722-2e15cb4b2903 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:32.680943 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b242fb7-c759-45c3-9722-2e15cb4b2903 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:32.681433 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b242fb7-c759-45c3-9722-2e15cb4b2903 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:32.682030 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b242fb7-c759-45c3-9722-2e15cb4b2903 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:32.695965 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b242fb7-c759-45c3-9722-2e15cb4b2903 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 09:05:32.706929 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-9b242fb7-c759-45c3-9722-2e15cb4b2903 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 14 09:05:32.706929 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:05:32.706929 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:05:32.706929 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:05:32.706929 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:05:32.706929 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:05:32.706929 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:05:32.706929 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:05:32.706929 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:05:32.706929 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:05:32.706929 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:05:32.706929 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:05:32.706929 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 09:05:32.706929 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 09:05:32.706929 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 09:05:32.706929 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 09:05:32.707914 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 09:05:32.707914 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 09:05:32.707914 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 09:05:32.707914 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 09:05:32.707914 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:05:32.707914 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:05:32.707914 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:05:32.707914 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:05:32.707914 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:05:32.707914 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:05:32.707914 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:05:32.707914 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 14 09:05:32.707914 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:05:32.707914 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:05:32.707914 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:05:32.707914 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:05:32.708780 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:05:32.708780 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:05:32.708780 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:05:32.708780 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:05:32.708780 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 09:05:32.708780 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 09:05:32.708780 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 09:05:32.708780 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 09:05:32.708780 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 09:05:32.708780 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 09:05:32.708780 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 14 09:05:32.708780 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 14 09:05:32.716809 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 453/917] 10.222.0.26 () {56 vars in 1120 bytes} [Thu Jul 14 09:05:32 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:05:32.718772 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b242fb7-c759-45c3-9722-2e15cb4b2903 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:32.730161 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c0ce08-caad-4aa4-bcbe-249cf2e8fe21 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:32.759645 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:32.765617 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4846336-a7f9-4978-9f36-4e5e350ba062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:32.766474 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4846336-a7f9-4978-9f36-4e5e350ba062 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:32.766753 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4846336-a7f9-4978-9f36-4e5e350ba062 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:32.767024 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4846336-a7f9-4978-9f36-4e5e350ba062 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:32.767860 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4846336-a7f9-4978-9f36-4e5e350ba062 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:32.769238 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4846336-a7f9-4978-9f36-4e5e350ba062 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:32.781102 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-a4846336-a7f9-4978-9f36-4e5e350ba062 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 14 09:05:32.781102 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:05:32.781102 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 09:05:32.781102 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 09:05:32.781102 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 09:05:32.781102 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 09:05:32.781102 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 09:05:32.781102 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 09:05:32.781102 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 14 09:05:32.781102 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 14 09:05:32.781102 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 09:05:32.781102 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 14 09:05:32.781102 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:05:32.781102 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:05:32.781102 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:05:32.781102 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:05:32.781102 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:05:32.782114 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:05:32.782114 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:05:32.782114 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:05:32.782114 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:05:32.782114 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:05:32.782114 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:05:32.782114 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 09:05:32.782114 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 09:05:32.782114 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 09:05:32.782114 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 09:05:32.782114 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:05:32.782114 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:05:32.782114 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:05:32.782114 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:05:32.782114 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:05:32.782114 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:05:32.783785 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:05:32.783785 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 14 09:05:32.783785 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:05:32.783785 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:05:32.783785 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:05:32.783785 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:05:32.783785 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:05:32.783785 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:05:32.783785 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:05:32.783785 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:05:32.783785 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 09:05:32.783785 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 09:05:32.783785 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 14 09:05:32.783785 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 14 09:05:32.783785 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 465/918] 10.222.0.26 () {56 vars in 1114 bytes} [Thu Jul 14 09:05:32 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 09:05:32.789577 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4846336-a7f9-4978-9f36-4e5e350ba062 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:32.796446 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:32.847678 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b242fb7-c759-45c3-9722-2e15cb4b2903 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:32.882895 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c85d1d-0799-41bc-9731-9e2cc42f9c7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:32.883594 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c85d1d-0799-41bc-9731-9e2cc42f9c7b None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:32.883874 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c85d1d-0799-41bc-9731-9e2cc42f9c7b None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:32.884144 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c85d1d-0799-41bc-9731-9e2cc42f9c7b None admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:32.884895 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c85d1d-0799-41bc-9731-9e2cc42f9c7b None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:05:32.885202 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c85d1d-0799-41bc-9731-9e2cc42f9c7b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:32.886573 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c85d1d-0799-41bc-9731-9e2cc42f9c7b None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:32.894734 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4846336-a7f9-4978-9f36-4e5e350ba062 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:32.901273 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 454/919] 10.222.0.26 () {56 vars in 1115 bytes} [Thu Jul 14 09:05:32 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:05:32.901273 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce323ed3-7c6f-4c21-bfa4-c4be4092ac0b None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:32.901273 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce323ed3-7c6f-4c21-bfa4-c4be4092ac0b None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:32.901273 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce323ed3-7c6f-4c21-bfa4-c4be4092ac0b None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:32.903412 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 455/920] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:05:32 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:05:32.904250 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce323ed3-7c6f-4c21-bfa4-c4be4092ac0b None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:32.937574 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1483c267-2238-4381-af47-1d3510706ff9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:32.938272 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1483c267-2238-4381-af47-1d3510706ff9 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:32.938560 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1483c267-2238-4381-af47-1d3510706ff9 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:32.938837 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1483c267-2238-4381-af47-1d3510706ff9 None admin] PATH_INFO: `/v3/services` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:32.939655 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1483c267-2238-4381-af47-1d3510706ff9 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:05:32.939965 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1483c267-2238-4381-af47-1d3510706ff9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:32.941621 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1483c267-2238-4381-af47-1d3510706ff9 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:32.951568 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 466/921] 10.222.0.26 () {56 vars in 1114 bytes} [Thu Jul 14 09:05:32 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 09:05:32.951568 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be386203-49c0-40d5-a280-50de7b22b63a None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:32.951568 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be386203-49c0-40d5-a280-50de7b22b63a None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:32.951568 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be386203-49c0-40d5-a280-50de7b22b63a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:32.967004 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-be386203-49c0-40d5-a280-50de7b22b63a None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:05:32.968221 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-be386203-49c0-40d5-a280-50de7b22b63a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:05:33.012620 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:33.014331 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce323ed3-7c6f-4c21-bfa4-c4be4092ac0b None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:33.036733 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:33.039629 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 467/922] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:05:32 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 09:05:33.040258 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be386203-49c0-40d5-a280-50de7b22b63a None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:33.070846 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb5abff-9f28-49eb-a44d-b50e0224d09b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:33.072175 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb5abff-9f28-49eb-a44d-b50e0224d09b None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:33.072941 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb5abff-9f28-49eb-a44d-b50e0224d09b None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:33.073652 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb5abff-9f28-49eb-a44d-b50e0224d09b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:33.087116 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb5abff-9f28-49eb-a44d-b50e0224d09b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:33.089440 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb5abff-9f28-49eb-a44d-b50e0224d09b None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:33.113491 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-0fb5abff-9f28-49eb-a44d-b50e0224d09b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 09:05:33.113491 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:05:33.113491 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:05:33.113491 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:05:33.113491 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:05:33.113491 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:05:33.113491 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:05:33.113491 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:05:33.113491 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:05:33.113491 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:05:33.113491 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:05:33.113491 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:05:33.113491 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 09:05:33.113491 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 09:05:33.113491 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 09:05:33.114549 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 09:05:33.114549 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:05:33.114549 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:05:33.114549 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:05:33.114549 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:05:33.114549 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:05:33.114549 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:05:33.114549 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:05:33.114549 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 14 09:05:33.114549 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:05:33.114549 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:05:33.114549 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:05:33.114549 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:05:33.114549 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:05:33.114549 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:05:33.114549 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:05:33.114549 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:05:33.115610 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 09:05:33.115610 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 09:05:33.115610 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 09:05:33.115610 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 14 09:05:33.115835 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 456/923] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jul 14 09:05:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:05:33.116714 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb5abff-9f28-49eb-a44d-b50e0224d09b None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:33.145234 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be386203-49c0-40d5-a280-50de7b22b63a None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:33.182244 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0087117e-e443-4049-bb40-0ea844b11787 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:33.183269 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0087117e-e443-4049-bb40-0ea844b11787 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:33.183545 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0087117e-e443-4049-bb40-0ea844b11787 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:33.183817 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0087117e-e443-4049-bb40-0ea844b11787 None admin] PATH_INFO: `/v3/services` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:33.184534 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0087117e-e443-4049-bb40-0ea844b11787 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:05:33.184843 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0087117e-e443-4049-bb40-0ea844b11787 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:33.186225 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0087117e-e443-4049-bb40-0ea844b11787 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:33.191693 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 468/924] 10.222.0.26 () {56 vars in 1114 bytes} [Thu Jul 14 09:05:33 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:05:33.192479 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0087117e-e443-4049-bb40-0ea844b11787 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:33.226547 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb5abff-9f28-49eb-a44d-b50e0224d09b None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:33.237483 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:33.263545 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea58f8e-3ffb-497f-95b5-b53e15af223f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:33.264338 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea58f8e-3ffb-497f-95b5-b53e15af223f None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:33.264616 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea58f8e-3ffb-497f-95b5-b53e15af223f None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:33.264889 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea58f8e-3ffb-497f-95b5-b53e15af223f None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:33.265546 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea58f8e-3ffb-497f-95b5-b53e15af223f None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:33.266922 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea58f8e-3ffb-497f-95b5-b53e15af223f None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:33.279008 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-7ea58f8e-3ffb-497f-95b5-b53e15af223f None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 14 09:05:33.279008 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:05:33.279008 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 09:05:33.279008 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 09:05:33.279008 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 09:05:33.279008 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 09:05:33.279008 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 09:05:33.279008 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 09:05:33.279008 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 14 09:05:33.279008 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 14 09:05:33.279008 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 09:05:33.279008 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 14 09:05:33.279008 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:05:33.279008 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:05:33.279008 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:05:33.279008 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:05:33.279008 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:05:33.280121 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:05:33.280121 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:05:33.280121 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:05:33.280121 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:05:33.280121 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:05:33.280121 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:05:33.280121 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 09:05:33.280121 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 09:05:33.280121 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 09:05:33.280121 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 09:05:33.280121 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:05:33.280121 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:05:33.280121 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:05:33.280121 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:05:33.280121 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:05:33.280121 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:05:33.280984 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:05:33.280984 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 14 09:05:33.280984 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:05:33.280984 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:05:33.280984 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:05:33.280984 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:05:33.280984 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:05:33.280984 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:05:33.280984 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:05:33.280984 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:05:33.280984 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 09:05:33.280984 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 09:05:33.280984 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 14 09:05:33.280984 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 14 09:05:33.280984 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 457/925] 10.222.0.26 () {56 vars in 1126 bytes} [Thu Jul 14 09:05:33 2022] GET /identity/v3/services/object-store => generated 93 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:05:33.281779 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea58f8e-3ffb-497f-95b5-b53e15af223f None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:33.305090 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0087117e-e443-4049-bb40-0ea844b11787 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:33.322593 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:33.339977 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b77d32-d14f-4354-9d5c-fe49adb6eaab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:33.340642 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b77d32-d14f-4354-9d5c-fe49adb6eaab None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:33.340952 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b77d32-d14f-4354-9d5c-fe49adb6eaab None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:33.341196 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b77d32-d14f-4354-9d5c-fe49adb6eaab None admin] PATH_INFO: `/v3/domains` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:33.341929 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b77d32-d14f-4354-9d5c-fe49adb6eaab None admin] RBAC: Adding query filter params (name=Default) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:05:33.342227 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b77d32-d14f-4354-9d5c-fe49adb6eaab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:33.343635 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b77d32-d14f-4354-9d5c-fe49adb6eaab None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:33.355928 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 469/926] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Jul 14 09:05:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:05:33.356530 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b77d32-d14f-4354-9d5c-fe49adb6eaab None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:33.398440 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea58f8e-3ffb-497f-95b5-b53e15af223f None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:33.433120 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d864a19b-8cfb-4b7c-a3af-2206d5f6c234 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:33.433787 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d864a19b-8cfb-4b7c-a3af-2206d5f6c234 None admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:33.434070 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d864a19b-8cfb-4b7c-a3af-2206d5f6c234 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:33.434342 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d864a19b-8cfb-4b7c-a3af-2206d5f6c234 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:33.435082 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d864a19b-8cfb-4b7c-a3af-2206d5f6c234 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:33.436569 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d864a19b-8cfb-4b7c-a3af-2206d5f6c234 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:33.455137 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 458/927] 10.222.0.26 () {60 vars in 1144 bytes} [Thu Jul 14 09:05:33 2022] POST /identity/v3/endpoints => generated 346 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 09:05:33.455912 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d864a19b-8cfb-4b7c-a3af-2206d5f6c234 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:33.458193 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:33.461842 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b77d32-d14f-4354-9d5c-fe49adb6eaab None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:33.506802 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1bbd10-79a9-4e3a-a2c5-56032211f9c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:33.507526 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1bbd10-79a9-4e3a-a2c5-56032211f9c9 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:33.507808 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1bbd10-79a9-4e3a-a2c5-56032211f9c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:33.508081 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1bbd10-79a9-4e3a-a2c5-56032211f9c9 None admin] PATH_INFO: `/v3/services` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:33.509139 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1bbd10-79a9-4e3a-a2c5-56032211f9c9 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:05:33.509139 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1bbd10-79a9-4e3a-a2c5-56032211f9c9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:33.510532 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1bbd10-79a9-4e3a-a2c5-56032211f9c9 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:33.521233 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 470/928] 10.222.0.26 () {56 vars in 1122 bytes} [Thu Jul 14 09:05:33 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:05:33.521233 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eedfe14-c968-430f-a6c5-c60bd2e58794 None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:33.521233 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eedfe14-c968-430f-a6c5-c60bd2e58794 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:33.521233 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eedfe14-c968-430f-a6c5-c60bd2e58794 None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:33.521233 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 471/929] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:05:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:05:33.522314 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eedfe14-c968-430f-a6c5-c60bd2e58794 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:33.585679 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:33.597392 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d864a19b-8cfb-4b7c-a3af-2206d5f6c234 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:33.636373 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5ee146-1505-40d8-b127-5c600e61bc2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:33.637198 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5ee146-1505-40d8-b127-5c600e61bc2d None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:33.637475 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5ee146-1505-40d8-b127-5c600e61bc2d None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:33.637748 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5ee146-1505-40d8-b127-5c600e61bc2d None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:33.645713 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5ee146-1505-40d8-b127-5c600e61bc2d None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:33.645883 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eedfe14-c968-430f-a6c5-c60bd2e58794 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:33.647687 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5ee146-1505-40d8-b127-5c600e61bc2d None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:33.663437 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-2a5ee146-1505-40d8-b127-5c600e61bc2d None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 14 09:05:33.663437 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:05:33.663437 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:05:33.663437 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:05:33.663437 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:05:33.663437 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:05:33.663437 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:05:33.663437 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:05:33.663437 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:05:33.663437 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:05:33.663437 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:05:33.663437 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:05:33.663437 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 09:05:33.663437 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 09:05:33.663437 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 09:05:33.663437 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 09:05:33.664468 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:05:33.664468 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:05:33.664468 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 09:05:33.664468 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 09:05:33.664468 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 09:05:33.664468 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 09:05:33.664468 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:05:33.664468 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:05:33.664468 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:05:33.664468 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:05:33.664468 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:05:33.664468 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 14 09:05:33.664468 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:05:33.664468 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:05:33.664468 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:05:33.664468 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:05:33.664468 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:05:33.664468 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:05:33.666054 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:05:33.666054 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:05:33.666054 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 09:05:33.666054 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 09:05:33.666054 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 09:05:33.666054 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 09:05:33.666054 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 09:05:33.666054 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 09:05:33.666054 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 14 09:05:33.666054 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 14 09:05:33.666054 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 459/930] 10.222.0.26 () {56 vars in 1117 bytes} [Thu Jul 14 09:05:33 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:05:33.666653 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaff9d1-5610-49b7-8371-fe12f790e931 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:33.666653 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaff9d1-5610-49b7-8371-fe12f790e931 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:33.666777 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaff9d1-5610-49b7-8371-fe12f790e931 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:33.684856 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-efaff9d1-5610-49b7-8371-fe12f790e931 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:05:33.686032 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-efaff9d1-5610-49b7-8371-fe12f790e931 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:05:33.687209 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235874ae-8547-4bbd-a8a7-35220593b4c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:33.688000 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235874ae-8547-4bbd-a8a7-35220593b4c5 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:33.688290 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235874ae-8547-4bbd-a8a7-35220593b4c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:33.688565 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235874ae-8547-4bbd-a8a7-35220593b4c5 None admin] PATH_INFO: `/v3/services` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:33.689840 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235874ae-8547-4bbd-a8a7-35220593b4c5 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:05:33.689840 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235874ae-8547-4bbd-a8a7-35220593b4c5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:33.690988 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235874ae-8547-4bbd-a8a7-35220593b4c5 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:33.696277 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 472/931] 10.222.0.26 () {56 vars in 1122 bytes} [Thu Jul 14 09:05:33 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:05:33.701352 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235874ae-8547-4bbd-a8a7-35220593b4c5 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:33.726880 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:33.763625 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 460/932] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:05:33 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 09:05:33.763625 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efaff9d1-5610-49b7-8371-fe12f790e931 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:33.820558 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235874ae-8547-4bbd-a8a7-35220593b4c5 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:33.849249 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:33.860222 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08dbf3ae-67db-4fff-93f5-ef0d4b0d74ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:33.860826 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dbf3ae-67db-4fff-93f5-ef0d4b0d74ca None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:33.861108 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dbf3ae-67db-4fff-93f5-ef0d4b0d74ca None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:33.861453 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dbf3ae-67db-4fff-93f5-ef0d4b0d74ca None admin] PATH_INFO: `/v3/users` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:33.862220 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08dbf3ae-67db-4fff-93f5-ef0d4b0d74ca None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:05:33.862529 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08dbf3ae-67db-4fff-93f5-ef0d4b0d74ca None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:33.863991 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08dbf3ae-67db-4fff-93f5-ef0d4b0d74ca None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:33.869908 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efaff9d1-5610-49b7-8371-fe12f790e931 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:33.887421 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 473/933] 10.222.0.26 () {56 vars in 1149 bytes} [Thu Jul 14 09:05:33 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:05:33.888318 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08dbf3ae-67db-4fff-93f5-ef0d4b0d74ca None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:33.907481 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0febe02a-5f93-486b-8595-54ac0a8cc35a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:33.908208 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0febe02a-5f93-486b-8595-54ac0a8cc35a None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:33.908483 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0febe02a-5f93-486b-8595-54ac0a8cc35a None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:33.908766 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0febe02a-5f93-486b-8595-54ac0a8cc35a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:33.909563 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0febe02a-5f93-486b-8595-54ac0a8cc35a None admin] RBAC: Adding query filter params (interface=public, service_id=ff4db0acaf2c4a219bdfe8c3341df62d, region_id=RegionOne) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:05:33.909876 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0febe02a-5f93-486b-8595-54ac0a8cc35a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:33.911274 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0febe02a-5f93-486b-8595-54ac0a8cc35a None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:33.916485 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 461/934] 10.222.0.26 () {56 vars in 1251 bytes} [Thu Jul 14 09:05:33 2022] GET /identity/v3/endpoints?service_id=ff4db0acaf2c4a219bdfe8c3341df62d&interface=public®ion_id=RegionOne => generated 194 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:05:33.917270 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0febe02a-5f93-486b-8595-54ac0a8cc35a None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:33.986939 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:34.006704 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08dbf3ae-67db-4fff-93f5-ef0d4b0d74ca None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:34.023001 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0febe02a-5f93-486b-8595-54ac0a8cc35a None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:34.044675 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29af6e75-0f99-4dad-969a-01ef10ce8d24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:34.045898 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29af6e75-0f99-4dad-969a-01ef10ce8d24 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:34.046418 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29af6e75-0f99-4dad-969a-01ef10ce8d24 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:34.046783 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29af6e75-0f99-4dad-969a-01ef10ce8d24 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:34.048102 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29af6e75-0f99-4dad-969a-01ef10ce8d24 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:34.049753 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29af6e75-0f99-4dad-969a-01ef10ce8d24 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:34.057267 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba7cc43-db7a-4eb9-8612-e5f229b10abe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:34.058171 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba7cc43-db7a-4eb9-8612-e5f229b10abe None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:34.058560 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba7cc43-db7a-4eb9-8612-e5f229b10abe None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:34.058925 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba7cc43-db7a-4eb9-8612-e5f229b10abe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:34.063453 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-29af6e75-0f99-4dad-969a-01ef10ce8d24 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 14 09:05:34.063453 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:05:34.063453 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 09:05:34.063453 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 09:05:34.063453 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 09:05:34.063453 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 09:05:34.063453 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 09:05:34.063453 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 09:05:34.063453 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 14 09:05:34.063453 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 14 09:05:34.063453 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 09:05:34.063453 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 14 09:05:34.063453 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:05:34.063453 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:05:34.063453 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:05:34.063453 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:05:34.064662 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:05:34.064662 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:05:34.064662 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:05:34.064662 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:05:34.064662 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:05:34.064662 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:05:34.064662 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:05:34.064662 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 09:05:34.064662 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 09:05:34.064662 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 09:05:34.064662 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 09:05:34.064662 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:05:34.064662 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:05:34.064662 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:05:34.064662 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:05:34.064662 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:05:34.070673 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:05:34.070673 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:05:34.070673 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 14 09:05:34.070673 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:05:34.070673 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:05:34.070673 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:05:34.070673 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:05:34.070673 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:05:34.070673 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:05:34.070673 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:05:34.070673 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:05:34.070673 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 09:05:34.070673 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 09:05:34.070673 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 14 09:05:34.070673 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 14 09:05:34.070673 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 474/935] 10.222.0.26 () {56 vars in 1132 bytes} [Thu Jul 14 09:05:33 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:05:34.071954 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29af6e75-0f99-4dad-969a-01ef10ce8d24 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:34.072380 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba7cc43-db7a-4eb9-8612-e5f229b10abe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:34.073893 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba7cc43-db7a-4eb9-8612-e5f229b10abe None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:34.080900 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:34.092564 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-cba7cc43-db7a-4eb9-8612-e5f229b10abe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 09:05:34.092564 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:05:34.092564 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:05:34.092564 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:05:34.092564 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:05:34.092564 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:05:34.092564 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:05:34.092564 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:05:34.092564 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:05:34.092564 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:05:34.092564 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:05:34.092564 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:05:34.092564 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 09:05:34.092564 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 09:05:34.092564 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 09:05:34.093705 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 09:05:34.093705 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:05:34.093705 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:05:34.093705 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:05:34.093705 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:05:34.093705 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:05:34.093705 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:05:34.093705 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:05:34.093705 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 14 09:05:34.093705 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:05:34.093705 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:05:34.093705 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:05:34.093705 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:05:34.093705 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:05:34.093705 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:05:34.093705 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:05:34.093705 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:05:34.094820 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 09:05:34.094820 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 09:05:34.094820 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 09:05:34.094820 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 14 09:05:34.100798 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 462/936] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jul 14 09:05:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:05:34.101734 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba7cc43-db7a-4eb9-8612-e5f229b10abe None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:34.180377 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29af6e75-0f99-4dad-969a-01ef10ce8d24 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:34.202675 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba7cc43-db7a-4eb9-8612-e5f229b10abe None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:34.215857 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bd3686-d64e-4e0e-8d10-c137794d7fcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:34.216690 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bd3686-d64e-4e0e-8d10-c137794d7fcb None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:34.217149 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bd3686-d64e-4e0e-8d10-c137794d7fcb None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:34.217448 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bd3686-d64e-4e0e-8d10-c137794d7fcb None admin] PATH_INFO: `/v3/services` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:34.218106 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bd3686-d64e-4e0e-8d10-c137794d7fcb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:34.219511 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bd3686-d64e-4e0e-8d10-c137794d7fcb None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:34.235217 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 475/937] 10.222.0.26 () {56 vars in 1087 bytes} [Thu Jul 14 09:05:34 2022] GET /identity/v3/services => generated 1497 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 09:05:34.235217 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bd3686-d64e-4e0e-8d10-c137794d7fcb None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:34.247257 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:34.252672 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e4cdc5-94bb-4a4b-a8e7-e8b6e64bc602 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:34.253492 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e4cdc5-94bb-4a4b-a8e7-e8b6e64bc602 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:34.254922 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e4cdc5-94bb-4a4b-a8e7-e8b6e64bc602 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:34.255087 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e4cdc5-94bb-4a4b-a8e7-e8b6e64bc602 None admin] PATH_INFO: `/v3/services` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:34.255887 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e4cdc5-94bb-4a4b-a8e7-e8b6e64bc602 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:05:34.257125 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e4cdc5-94bb-4a4b-a8e7-e8b6e64bc602 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:34.258803 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e4cdc5-94bb-4a4b-a8e7-e8b6e64bc602 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:34.274937 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 463/938] 10.222.0.26 () {56 vars in 1126 bytes} [Thu Jul 14 09:05:34 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:05:34.276295 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e4cdc5-94bb-4a4b-a8e7-e8b6e64bc602 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:34.353993 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:34.397701 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bd3686-d64e-4e0e-8d10-c137794d7fcb None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:34.405428 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e4cdc5-94bb-4a4b-a8e7-e8b6e64bc602 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:34.438104 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3cadcd-4a78-46f4-9d17-a4e5fe9b9dda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:34.438804 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3cadcd-4a78-46f4-9d17-a4e5fe9b9dda None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:34.439134 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3cadcd-4a78-46f4-9d17-a4e5fe9b9dda None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:34.439447 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3cadcd-4a78-46f4-9d17-a4e5fe9b9dda None admin] PATH_INFO: `/v3/domains` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:34.440170 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3cadcd-4a78-46f4-9d17-a4e5fe9b9dda None admin] RBAC: Adding query filter params (name=Default) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:05:34.440472 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3cadcd-4a78-46f4-9d17-a4e5fe9b9dda None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:34.441843 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3cadcd-4a78-46f4-9d17-a4e5fe9b9dda None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:34.446266 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960f6467-0fee-4fa4-89b1-087faa1416d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:34.446939 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960f6467-0fee-4fa4-89b1-087faa1416d1 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:34.447242 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960f6467-0fee-4fa4-89b1-087faa1416d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:34.447526 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960f6467-0fee-4fa4-89b1-087faa1416d1 None admin] PATH_INFO: `/v3/services` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:34.448238 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960f6467-0fee-4fa4-89b1-087faa1416d1 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:05:34.448544 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960f6467-0fee-4fa4-89b1-087faa1416d1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:34.450499 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960f6467-0fee-4fa4-89b1-087faa1416d1 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:34.454526 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 476/939] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Jul 14 09:05:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:05:34.457643 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 464/940] 10.222.0.26 () {56 vars in 1126 bytes} [Thu Jul 14 09:05:34 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:05:34.462746 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3cadcd-4a78-46f4-9d17-a4e5fe9b9dda None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:34.464037 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960f6467-0fee-4fa4-89b1-087faa1416d1 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:34.535824 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:34.568729 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3cadcd-4a78-46f4-9d17-a4e5fe9b9dda None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:34.581284 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960f6467-0fee-4fa4-89b1-087faa1416d1 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:34.605711 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c88488-e9c1-493f-bceb-dcd60c32241a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:34.606219 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c88488-e9c1-493f-bceb-dcd60c32241a None admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:34.606502 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c88488-e9c1-493f-bceb-dcd60c32241a None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:34.606777 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c88488-e9c1-493f-bceb-dcd60c32241a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:34.607573 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c88488-e9c1-493f-bceb-dcd60c32241a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:34.608954 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c88488-e9c1-493f-bceb-dcd60c32241a None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:34.613120 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:34.624615 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff29cb7-c8fa-4ae7-bdfb-bf97d5f677a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:34.625892 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff29cb7-c8fa-4ae7-bdfb-bf97d5f677a1 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:34.626447 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff29cb7-c8fa-4ae7-bdfb-bf97d5f677a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:34.627008 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff29cb7-c8fa-4ae7-bdfb-bf97d5f677a1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:34.635273 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 477/941] 10.222.0.26 () {60 vars in 1144 bytes} [Thu Jul 14 09:05:34 2022] POST /identity/v3/endpoints => generated 347 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 09:05:34.640997 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff29cb7-c8fa-4ae7-bdfb-bf97d5f677a1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:34.642962 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff29cb7-c8fa-4ae7-bdfb-bf97d5f677a1 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:34.659572 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-aff29cb7-c8fa-4ae7-bdfb-bf97d5f677a1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 09:05:34.659572 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:05:34.659572 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:05:34.659572 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:05:34.659572 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:05:34.659572 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:05:34.659572 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:05:34.659572 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:05:34.659572 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:05:34.659572 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:05:34.659572 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:05:34.659572 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:05:34.659572 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 09:05:34.659572 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 09:05:34.659572 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 09:05:34.660590 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 09:05:34.660590 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:05:34.660590 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:05:34.660590 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:05:34.660590 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:05:34.660590 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:05:34.660590 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:05:34.660590 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:05:34.660590 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 14 09:05:34.660590 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:05:34.660590 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:05:34.660590 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:05:34.660590 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:05:34.660590 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:05:34.660590 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:05:34.660590 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:05:34.660590 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:05:34.661486 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 09:05:34.661486 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 09:05:34.661486 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 09:05:34.661486 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 09:05:34.661486 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 09:05:34.661486 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 09:05:34.661486 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 09:05:34.661486 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 14 09:05:34.664265 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 465/942] 10.222.0.26 () {56 vars in 1111 bytes} [Thu Jul 14 09:05:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 09:05:34.669803 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c88488-e9c1-493f-bceb-dcd60c32241a None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:34.794300 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c88488-e9c1-493f-bceb-dcd60c32241a None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:34.820714 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:34.834047 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c593fb-bbff-4128-8b1f-45185ba04f09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:34.834719 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c593fb-bbff-4128-8b1f-45185ba04f09 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:34.835002 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c593fb-bbff-4128-8b1f-45185ba04f09 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:34.835302 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c593fb-bbff-4128-8b1f-45185ba04f09 None admin] PATH_INFO: `/v3/projects` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:34.836154 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c593fb-bbff-4128-8b1f-45185ba04f09 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:05:34.836460 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c593fb-bbff-4128-8b1f-45185ba04f09 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:34.837944 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c593fb-bbff-4128-8b1f-45185ba04f09 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:34.850398 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 478/943] 10.222.0.26 () {56 vars in 1148 bytes} [Thu Jul 14 09:05:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 09:05:34.856279 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff29cb7-c8fa-4ae7-bdfb-bf97d5f677a1 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:34.961860 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff29cb7-c8fa-4ae7-bdfb-bf97d5f677a1 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:34.997115 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cd5c20-a857-4ade-b4c1-6b55466c5c6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:34.998006 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cd5c20-a857-4ade-b4c1-6b55466c5c6a None admin] REQUEST_METHOD: `PUT` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:34.998286 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cd5c20-a857-4ade-b4c1-6b55466c5c6a None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:34.998555 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cd5c20-a857-4ade-b4c1-6b55466c5c6a None admin] PATH_INFO: `/v3/projects/33bf210d015847439655b9943d1b0261/users/7d6267ede8d64e2abe1f4058a9797fb7/roles/70488f05098d4ce09e2fb03aaaea4566` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:35.000790 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26cd5c20-a857-4ade-b4c1-6b55466c5c6a None admin] RBAC: Authorizing `identity:create_grant(project_id=33bf210d015847439655b9943d1b0261, user_id=7d6267ede8d64e2abe1f4058a9797fb7, role_id=70488f05098d4ce09e2fb03aaaea4566)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:35.002273 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26cd5c20-a857-4ade-b4c1-6b55466c5c6a None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:35.018482 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 466/944] 10.222.0.26 () {58 vars in 1439 bytes} [Thu Jul 14 09:05:34 2022] PUT /identity/v3/projects/33bf210d015847439655b9943d1b0261/users/7d6267ede8d64e2abe1f4058a9797fb7/roles/70488f05098d4ce09e2fb03aaaea4566 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:05:35.523906 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd7bed3-3c48-4768-b7b1-39d6a2a49bbe None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:35.524948 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd7bed3-3c48-4768-b7b1-39d6a2a49bbe None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:35.525437 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd7bed3-3c48-4768-b7b1-39d6a2a49bbe None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:35.527285 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 479/945] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:05:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:05:35.532622 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d52092-b9bf-4153-94b5-db11fbe3da28 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:35.533040 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d52092-b9bf-4153-94b5-db11fbe3da28 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:35.533150 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d52092-b9bf-4153-94b5-db11fbe3da28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:35.550510 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-35d52092-b9bf-4153-94b5-db11fbe3da28 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:05:35.551772 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-35d52092-b9bf-4153-94b5-db11fbe3da28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:05:35.598468 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:35.626981 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 467/946] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:05:35 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 09:05:35.650174 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd7bed3-3c48-4768-b7b1-39d6a2a49bbe None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:35.759913 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd7bed3-3c48-4768-b7b1-39d6a2a49bbe None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:35.794542 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8279b546-d0e6-49bd-a6f6-3104cc65d307 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:35.795127 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8279b546-d0e6-49bd-a6f6-3104cc65d307 None admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:35.795433 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8279b546-d0e6-49bd-a6f6-3104cc65d307 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:35.795708 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8279b546-d0e6-49bd-a6f6-3104cc65d307 None admin] PATH_INFO: `/v3/projects` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:35.796426 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8279b546-d0e6-49bd-a6f6-3104cc65d307 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:35.797923 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8279b546-d0e6-49bd-a6f6-3104cc65d307 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:35.799368 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: WARNING keystone.server.flask.common [None req-8279b546-d0e6-49bd-a6f6-3104cc65d307 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 14 09:05:35.818255 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.sql.core [None req-8279b546-d0e6-49bd-a6f6-3104cc65d307 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 14 09:05:35.818255 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 14 09:05:35.818255 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [parameters: {'id': 'f93dd83facb145789c4651f67aff1e33', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 14 09:05:35.818255 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80382) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 14 09:05:35.836003 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-8279b546-d0e6-49bd-a6f6-3104cc65d307 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id f93dd83facb145789c4651f67aff1e33.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id f93dd83facb145789c4651f67aff1e33. Jul 14 09:05:35.836003 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:05:35.836003 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 14 09:05:35.836003 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 14 09:05:35.836003 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 14 09:05:35.836003 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 14 09:05:35.836003 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 14 09:05:35.836003 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application result = self._query(query) Jul 14 09:05:35.836003 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 14 09:05:35.836003 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application conn.query(q) Jul 14 09:05:35.836003 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 14 09:05:35.836003 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 14 09:05:35.836003 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 14 09:05:35.836911 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application result.read() Jul 14 09:05:35.836911 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 14 09:05:35.836911 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 14 09:05:35.836911 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 14 09:05:35.836911 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 14 09:05:35.836911 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 14 09:05:35.836911 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 14 09:05:35.836911 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 14 09:05:35.836911 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 14 09:05:35.836911 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 14 09:05:35.836911 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 14 09:05:35.836911 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 14 09:05:35.836911 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 14 09:05:35.836911 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:05:35.836911 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 14 09:05:35.836911 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 14 09:05:35.836911 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jul 14 09:05:35.836911 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return project_ref.to_dict() Jul 14 09:05:35.838357 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 14 09:05:35.838357 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application next(self.gen) Jul 14 09:05:35.838357 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 14 09:05:35.838357 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application yield resource Jul 14 09:05:35.838357 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 14 09:05:35.838357 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application next(self.gen) Jul 14 09:05:35.838357 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 14 09:05:35.838357 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application self.session.rollback() Jul 14 09:05:35.838357 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 14 09:05:35.838357 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application self.force_reraise() Jul 14 09:05:35.838357 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 14 09:05:35.838357 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise self.value Jul 14 09:05:35.838357 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 14 09:05:35.838357 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 14 09:05:35.838357 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 14 09:05:35.838357 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application session.commit() Jul 14 09:05:35.838357 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 14 09:05:35.839328 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 14 09:05:35.839328 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 14 09:05:35.839328 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application self._prepare_impl() Jul 14 09:05:35.839328 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 14 09:05:35.839328 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application self.session.flush() Jul 14 09:05:35.839328 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 14 09:05:35.839328 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application self._flush(objects) Jul 14 09:05:35.839328 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 14 09:05:35.839328 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 14 09:05:35.839328 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 14 09:05:35.839328 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application compat.raise_( Jul 14 09:05:35.839328 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 14 09:05:35.839328 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception Jul 14 09:05:35.839328 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 14 09:05:35.839328 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application flush_context.execute() Jul 14 09:05:35.839328 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 14 09:05:35.839328 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rec.execute(self) Jul 14 09:05:35.840333 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 14 09:05:35.840333 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 14 09:05:35.840333 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 14 09:05:35.840333 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 14 09:05:35.840333 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 14 09:05:35.840333 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 14 09:05:35.840333 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 14 09:05:35.840333 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 14 09:05:35.840333 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 14 09:05:35.840333 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 14 09:05:35.840333 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 14 09:05:35.840333 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 14 09:05:35.840333 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 14 09:05:35.840333 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 14 09:05:35.840333 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 14 09:05:35.841145 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 14 09:05:35.841145 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 14 09:05:35.841145 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception Jul 14 09:05:35.841145 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 14 09:05:35.841145 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 14 09:05:35.841145 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 14 09:05:35.841145 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 14 09:05:35.841145 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 14 09:05:35.841145 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application result = self._query(query) Jul 14 09:05:35.841145 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 14 09:05:35.841145 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application conn.query(q) Jul 14 09:05:35.841145 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 14 09:05:35.841145 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 14 09:05:35.841145 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 14 09:05:35.841145 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application result.read() Jul 14 09:05:35.841145 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 14 09:05:35.841145 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 14 09:05:35.842056 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 14 09:05:35.842056 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 14 09:05:35.842056 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 14 09:05:35.842056 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 14 09:05:35.842056 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 14 09:05:35.842056 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 14 09:05:35.842056 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 14 09:05:35.842056 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 14 09:05:35.842056 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [parameters: {'id': 'f93dd83facb145789c4651f67aff1e33', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 14 09:05:35.842056 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 14 09:05:35.842056 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 14 09:05:35.842056 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 09:05:35.842056 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 14 09:05:35.842056 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:05:35.842056 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jul 14 09:05:35.842882 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jul 14 09:05:35.842882 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 14 09:05:35.842882 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 14 09:05:35.842882 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jul 14 09:05:35.842882 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 14 09:05:35.842882 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 09:05:35.842882 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 14 09:05:35.842882 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:05:35.842882 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:05:35.842882 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:05:35.842882 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:05:35.842882 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:05:35.842882 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:05:35.842882 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:05:35.842882 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:05:35.842882 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:05:35.842882 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:05:35.845653 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:05:35.845653 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jul 14 09:05:35.845653 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jul 14 09:05:35.845653 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:05:35.845653 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:05:35.845653 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jul 14 09:05:35.845653 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.Conflict( Jul 14 09:05:35.845653 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id f93dd83facb145789c4651f67aff1e33. Jul 14 09:05:35.845653 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 14 09:05:35.848722 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 480/947] 10.222.0.26 () {60 vars in 1140 bytes} [Thu Jul 14 09:05:35 2022] POST /identity/v3/projects => generated 262 bytes in 199 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:05:35.854644 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d52092-b9bf-4153-94b5-db11fbe3da28 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:35.962125 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d52092-b9bf-4153-94b5-db11fbe3da28 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:35.997625 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84df854-f89d-4d6b-a78d-7a141ab9db5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:35.998832 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84df854-f89d-4d6b-a78d-7a141ab9db5f None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:35.999349 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84df854-f89d-4d6b-a78d-7a141ab9db5f None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:35.999870 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84df854-f89d-4d6b-a78d-7a141ab9db5f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:36.007268 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84df854-f89d-4d6b-a78d-7a141ab9db5f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:36.008971 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84df854-f89d-4d6b-a78d-7a141ab9db5f None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:36.023359 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-e84df854-f89d-4d6b-a78d-7a141ab9db5f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 09:05:36.023359 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:05:36.023359 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:05:36.023359 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:05:36.023359 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:05:36.023359 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:05:36.023359 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:05:36.023359 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:05:36.023359 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:05:36.023359 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:05:36.023359 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:05:36.023359 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:05:36.023359 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 09:05:36.023359 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 09:05:36.023359 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 09:05:36.024321 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 09:05:36.024321 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:05:36.024321 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:05:36.024321 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:05:36.024321 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:05:36.024321 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:05:36.024321 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:05:36.024321 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:05:36.024321 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 14 09:05:36.024321 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:05:36.024321 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:05:36.024321 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:05:36.024321 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:05:36.024321 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:05:36.024321 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:05:36.024321 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:05:36.024321 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:05:36.025294 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 09:05:36.025294 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 09:05:36.025294 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 09:05:36.025294 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 09:05:36.025294 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 09:05:36.025294 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 09:05:36.025294 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 09:05:36.025294 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 14 09:05:36.028155 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 468/948] 10.222.0.26 () {56 vars in 1111 bytes} [Thu Jul 14 09:05:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:05:36.035015 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8279b546-d0e6-49bd-a6f6-3104cc65d307 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:36.081610 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:36.145317 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8279b546-d0e6-49bd-a6f6-3104cc65d307 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:36.187241 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2383826-f61a-4e68-88dc-7bb32f2787cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:36.187913 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2383826-f61a-4e68-88dc-7bb32f2787cb None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:36.188191 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2383826-f61a-4e68-88dc-7bb32f2787cb None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:36.188531 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2383826-f61a-4e68-88dc-7bb32f2787cb None admin] PATH_INFO: `/v3/projects` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:36.189297 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2383826-f61a-4e68-88dc-7bb32f2787cb None admin] RBAC: Adding query filter params (name=service) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:05:36.189598 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2383826-f61a-4e68-88dc-7bb32f2787cb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:36.191275 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2383826-f61a-4e68-88dc-7bb32f2787cb None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:36.213100 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 481/949] 10.222.0.26 () {56 vars in 1112 bytes} [Thu Jul 14 09:05:36 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:05:36.336856 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835669b2-9ec8-411a-b622-721db9a2692a None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:36.337070 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835669b2-9ec8-411a-b622-721db9a2692a None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:36.337381 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835669b2-9ec8-411a-b622-721db9a2692a None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:36.339127 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 469/950] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:05:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:05:36.345412 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fda98e2-237d-40f7-8f64-2530cd433e1d None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:36.345864 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fda98e2-237d-40f7-8f64-2530cd433e1d None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:36.346274 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fda98e2-237d-40f7-8f64-2530cd433e1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:36.364822 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-5fda98e2-237d-40f7-8f64-2530cd433e1d None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:05:36.366319 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-5fda98e2-237d-40f7-8f64-2530cd433e1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:05:36.437255 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 482/951] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:05:36 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 09:05:36.462552 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835669b2-9ec8-411a-b622-721db9a2692a None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:36.506546 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:36.575347 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835669b2-9ec8-411a-b622-721db9a2692a None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:36.610138 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdb7a0c-8ce0-4b37-969e-02101bf0a201 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:36.610873 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdb7a0c-8ce0-4b37-969e-02101bf0a201 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:36.611159 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdb7a0c-8ce0-4b37-969e-02101bf0a201 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:36.611464 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdb7a0c-8ce0-4b37-969e-02101bf0a201 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:36.612130 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdb7a0c-8ce0-4b37-969e-02101bf0a201 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:36.613506 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdb7a0c-8ce0-4b37-969e-02101bf0a201 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:36.625880 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-3cdb7a0c-8ce0-4b37-969e-02101bf0a201 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 14 09:05:36.625880 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:05:36.625880 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 09:05:36.625880 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 09:05:36.625880 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 09:05:36.625880 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 09:05:36.625880 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 09:05:36.625880 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 09:05:36.625880 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 14 09:05:36.625880 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 14 09:05:36.625880 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 09:05:36.625880 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 14 09:05:36.625880 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:05:36.625880 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:05:36.625880 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:05:36.625880 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:05:36.625880 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:05:36.626959 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:05:36.626959 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:05:36.626959 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:05:36.626959 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:05:36.626959 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:05:36.626959 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:05:36.626959 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 09:05:36.626959 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 09:05:36.626959 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 09:05:36.626959 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 09:05:36.626959 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:05:36.626959 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:05:36.626959 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:05:36.626959 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:05:36.626959 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:05:36.626959 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:05:36.628740 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:05:36.628740 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 14 09:05:36.628740 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:05:36.628740 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:05:36.628740 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:05:36.628740 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:05:36.628740 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:05:36.628740 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:05:36.628740 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:05:36.628740 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:05:36.628740 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 09:05:36.628740 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 09:05:36.628740 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 14 09:05:36.628740 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 14 09:05:36.628740 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 470/952] 10.222.0.26 () {56 vars in 1126 bytes} [Thu Jul 14 09:05:36 2022] GET /identity/v3/services/object-store => generated 93 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:05:36.633582 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fda98e2-237d-40f7-8f64-2530cd433e1d None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:36.662674 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:36.740567 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fda98e2-237d-40f7-8f64-2530cd433e1d None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:36.780654 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ecedb7-099d-40ab-9d1e-6a6668a2ab95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:36.781646 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ecedb7-099d-40ab-9d1e-6a6668a2ab95 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:36.781956 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ecedb7-099d-40ab-9d1e-6a6668a2ab95 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:36.782229 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ecedb7-099d-40ab-9d1e-6a6668a2ab95 None admin] PATH_INFO: `/v3/services` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:36.782943 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ecedb7-099d-40ab-9d1e-6a6668a2ab95 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:05:36.783275 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ecedb7-099d-40ab-9d1e-6a6668a2ab95 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:36.784658 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ecedb7-099d-40ab-9d1e-6a6668a2ab95 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:36.790318 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 483/953] 10.222.0.26 () {56 vars in 1122 bytes} [Thu Jul 14 09:05:36 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:05:36.796035 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdb7a0c-8ce0-4b37-969e-02101bf0a201 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:36.826837 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9240447f-6d1d-4451-95a5-8649efbac51c None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:36.827122 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9240447f-6d1d-4451-95a5-8649efbac51c None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:36.827432 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9240447f-6d1d-4451-95a5-8649efbac51c None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:36.829698 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 484/954] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:05:36 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 09:05:36.834768 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93554ff0-3a08-47da-aa46-89660213cadd None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:36.835056 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93554ff0-3a08-47da-aa46-89660213cadd None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:36.835373 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93554ff0-3a08-47da-aa46-89660213cadd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:36.853051 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-93554ff0-3a08-47da-aa46-89660213cadd None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:05:36.854225 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-93554ff0-3a08-47da-aa46-89660213cadd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:05:36.912260 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:36.916395 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdb7a0c-8ce0-4b37-969e-02101bf0a201 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:36.931457 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 485/955] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:05:36 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 09:05:36.946883 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93554ff0-3a08-47da-aa46-89660213cadd None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:36.956745 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1946be57-36b3-4cd1-95cc-8b7a34a3c4d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:36.958194 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1946be57-36b3-4cd1-95cc-8b7a34a3c4d4 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:36.959067 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1946be57-36b3-4cd1-95cc-8b7a34a3c4d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:36.959588 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1946be57-36b3-4cd1-95cc-8b7a34a3c4d4 None admin] PATH_INFO: `/v3/services` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:36.960772 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1946be57-36b3-4cd1-95cc-8b7a34a3c4d4 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:05:36.961254 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1946be57-36b3-4cd1-95cc-8b7a34a3c4d4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:36.962954 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1946be57-36b3-4cd1-95cc-8b7a34a3c4d4 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:36.970247 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 471/956] 10.222.0.26 () {56 vars in 1122 bytes} [Thu Jul 14 09:05:36 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:05:36.975473 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:36.976268 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1946be57-36b3-4cd1-95cc-8b7a34a3c4d4 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:37.075629 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93554ff0-3a08-47da-aa46-89660213cadd None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:37.106269 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1946be57-36b3-4cd1-95cc-8b7a34a3c4d4 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:37.118771 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22eac95-303a-4f6b-a400-74911c05d413 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:37.119900 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22eac95-303a-4f6b-a400-74911c05d413 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:37.120409 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22eac95-303a-4f6b-a400-74911c05d413 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:37.120876 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22eac95-303a-4f6b-a400-74911c05d413 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:37.121728 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22eac95-303a-4f6b-a400-74911c05d413 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:37.123322 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22eac95-303a-4f6b-a400-74911c05d413 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:37.136348 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-d22eac95-303a-4f6b-a400-74911c05d413 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 14 09:05:37.136348 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:05:37.136348 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 09:05:37.136348 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 09:05:37.136348 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 09:05:37.136348 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 09:05:37.136348 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 09:05:37.136348 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 09:05:37.136348 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 14 09:05:37.136348 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 14 09:05:37.136348 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 09:05:37.136348 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 14 09:05:37.136348 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:05:37.136348 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:05:37.136348 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:05:37.136348 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:05:37.136348 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:05:37.137494 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:05:37.137494 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:05:37.137494 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:05:37.137494 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:05:37.137494 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:05:37.137494 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:05:37.137494 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 09:05:37.137494 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 09:05:37.137494 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 09:05:37.137494 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 09:05:37.137494 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:05:37.137494 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:05:37.137494 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:05:37.137494 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:05:37.137494 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:05:37.137494 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:05:37.138412 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:05:37.138412 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 14 09:05:37.138412 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:05:37.138412 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:05:37.138412 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:05:37.138412 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:05:37.138412 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:05:37.138412 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:05:37.138412 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:05:37.138412 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:05:37.138412 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 09:05:37.138412 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 09:05:37.138412 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 14 09:05:37.138412 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 14 09:05:37.141362 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 486/957] 10.222.0.26 () {56 vars in 1111 bytes} [Thu Jul 14 09:05:36 2022] GET /identity/v3/services/compute => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 09:05:37.142919 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8285d41c-87b4-42bd-8fec-461a4a39e292 None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:37.143428 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8285d41c-87b4-42bd-8fec-461a4a39e292 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:37.143906 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8285d41c-87b4-42bd-8fec-461a4a39e292 None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:37.145754 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 487/958] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:05:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:05:37.149553 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8285d41c-87b4-42bd-8fec-461a4a39e292 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:37.150186 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b589de-1a8f-443f-bd05-868d6ba4c134 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:37.150860 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b589de-1a8f-443f-bd05-868d6ba4c134 None admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:37.151140 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b589de-1a8f-443f-bd05-868d6ba4c134 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:37.151444 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b589de-1a8f-443f-bd05-868d6ba4c134 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:37.152491 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b589de-1a8f-443f-bd05-868d6ba4c134 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:37.154823 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b589de-1a8f-443f-bd05-868d6ba4c134 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:37.177414 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 472/959] 10.222.0.26 () {60 vars in 1144 bytes} [Thu Jul 14 09:05:36 2022] POST /identity/v3/endpoints => generated 349 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 09:05:37.178583 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b0cbc3-dc77-4fd6-86a3-61dd9cecb937 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:37.178871 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b0cbc3-dc77-4fd6-86a3-61dd9cecb937 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:37.179144 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b0cbc3-dc77-4fd6-86a3-61dd9cecb937 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:37.206524 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-43b0cbc3-dc77-4fd6-86a3-61dd9cecb937 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:05:37.208119 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-43b0cbc3-dc77-4fd6-86a3-61dd9cecb937 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:05:37.232350 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:37.261332 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:37.287470 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8285d41c-87b4-42bd-8fec-461a4a39e292 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:37.319393 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 473/960] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:05:37 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 09:05:37.341213 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f814910-3933-4010-aaec-859a7ff72df7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:37.341894 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f814910-3933-4010-aaec-859a7ff72df7 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:37.342179 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f814910-3933-4010-aaec-859a7ff72df7 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:37.342453 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f814910-3933-4010-aaec-859a7ff72df7 None admin] PATH_INFO: `/v3/services` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:37.343173 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f814910-3933-4010-aaec-859a7ff72df7 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:05:37.343504 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f814910-3933-4010-aaec-859a7ff72df7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:37.344910 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f814910-3933-4010-aaec-859a7ff72df7 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:37.346550 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b0cbc3-dc77-4fd6-86a3-61dd9cecb937 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:37.350345 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 488/961] 10.222.0.26 () {56 vars in 1112 bytes} [Thu Jul 14 09:05:37 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 09:05:37.355797 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f814910-3933-4010-aaec-859a7ff72df7 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:37.466456 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f814910-3933-4010-aaec-859a7ff72df7 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:37.471361 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b0cbc3-dc77-4fd6-86a3-61dd9cecb937 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:37.513871 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643298fa-f669-4d89-b515-3ec1424f9816 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:37.514690 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643298fa-f669-4d89-b515-3ec1424f9816 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:37.514976 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643298fa-f669-4d89-b515-3ec1424f9816 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:37.515283 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643298fa-f669-4d89-b515-3ec1424f9816 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:37.517774 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069af969-7d9d-4c7b-be58-1883586ce14c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:37.518984 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069af969-7d9d-4c7b-be58-1883586ce14c None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:37.519479 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069af969-7d9d-4c7b-be58-1883586ce14c None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:37.519927 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069af969-7d9d-4c7b-be58-1883586ce14c None admin] PATH_INFO: `/v3/services` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:37.521108 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069af969-7d9d-4c7b-be58-1883586ce14c None admin] RBAC: Adding query filter params (type=compute) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:05:37.521687 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069af969-7d9d-4c7b-be58-1883586ce14c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:37.523566 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069af969-7d9d-4c7b-be58-1883586ce14c None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:37.526692 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643298fa-f669-4d89-b515-3ec1424f9816 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 09:05:37.535496 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 489/962] 10.222.0.26 () {56 vars in 1112 bytes} [Thu Jul 14 09:05:37 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 09:05:37.536171 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-643298fa-f669-4d89-b515-3ec1424f9816 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 14 09:05:37.536171 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:05:37.536171 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:05:37.536171 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:05:37.536171 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:05:37.536171 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:05:37.536171 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:05:37.536171 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:05:37.536171 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:05:37.536171 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:05:37.536171 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:05:37.536171 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:05:37.536171 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 09:05:37.536171 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 09:05:37.536171 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 09:05:37.536171 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 09:05:37.537306 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 09:05:37.537306 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 09:05:37.537306 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 09:05:37.537306 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 09:05:37.537306 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:05:37.537306 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:05:37.537306 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:05:37.537306 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:05:37.537306 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:05:37.537306 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:05:37.537306 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:05:37.537306 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 14 09:05:37.537306 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:05:37.537306 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:05:37.537306 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:05:37.537306 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:05:37.538387 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:05:37.538387 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:05:37.538387 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:05:37.538387 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:05:37.538387 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 09:05:37.538387 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 09:05:37.538387 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 09:05:37.538387 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 09:05:37.538387 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 09:05:37.538387 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 09:05:37.538387 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 14 09:05:37.538387 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 14 09:05:37.540257 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 474/963] 10.222.0.26 () {56 vars in 1120 bytes} [Thu Jul 14 09:05:37 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 09:05:37.552975 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069af969-7d9d-4c7b-be58-1883586ce14c None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:37.568039 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:37.665351 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069af969-7d9d-4c7b-be58-1883586ce14c None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:37.702003 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b989cebc-a60e-4adb-8a1f-31389612c2f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:37.702937 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b989cebc-a60e-4adb-8a1f-31389612c2f3 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:37.703387 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b989cebc-a60e-4adb-8a1f-31389612c2f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:37.703877 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b989cebc-a60e-4adb-8a1f-31389612c2f3 None admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:37.704838 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b989cebc-a60e-4adb-8a1f-31389612c2f3 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:05:37.705291 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b989cebc-a60e-4adb-8a1f-31389612c2f3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:37.706844 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b989cebc-a60e-4adb-8a1f-31389612c2f3 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:37.716896 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 490/964] 10.222.0.26 () {56 vars in 1115 bytes} [Thu Jul 14 09:05:37 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:05:37.722692 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643298fa-f669-4d89-b515-3ec1424f9816 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:37.736471 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:37.832079 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643298fa-f669-4d89-b515-3ec1424f9816 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:37.871488 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380db722-03af-42da-b732-f883e508d691 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:37.872542 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380db722-03af-42da-b732-f883e508d691 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:37.873009 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380db722-03af-42da-b732-f883e508d691 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:37.873442 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380db722-03af-42da-b732-f883e508d691 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:37.884860 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380db722-03af-42da-b732-f883e508d691 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:37.886555 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380db722-03af-42da-b732-f883e508d691 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:37.905256 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-380db722-03af-42da-b732-f883e508d691 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 09:05:37.905256 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:05:37.905256 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:05:37.905256 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:05:37.905256 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:05:37.905256 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:05:37.905256 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:05:37.905256 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:05:37.905256 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:05:37.905256 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:05:37.905256 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:05:37.905256 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:05:37.905256 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 09:05:37.905256 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 09:05:37.905256 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 09:05:37.906801 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 09:05:37.906801 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:05:37.906801 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:05:37.906801 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:05:37.906801 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:05:37.906801 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:05:37.906801 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:05:37.906801 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:05:37.906801 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 14 09:05:37.906801 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:05:37.906801 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:05:37.906801 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:05:37.906801 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:05:37.906801 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:05:37.906801 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:05:37.906801 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:05:37.906801 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:05:37.907854 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 09:05:37.907854 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 09:05:37.907854 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 09:05:37.907854 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 14 09:05:37.909725 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 475/965] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jul 14 09:05:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:05:37.915415 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b989cebc-a60e-4adb-8a1f-31389612c2f3 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:37.987414 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:38.022580 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b989cebc-a60e-4adb-8a1f-31389612c2f3 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:38.057543 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ad0da2-90c4-4d9d-a417-55816c14c8bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:38.058220 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ad0da2-90c4-4d9d-a417-55816c14c8bd None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:38.058507 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ad0da2-90c4-4d9d-a417-55816c14c8bd None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:38.058783 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ad0da2-90c4-4d9d-a417-55816c14c8bd None admin] PATH_INFO: `/v3/domains` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:38.059554 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ad0da2-90c4-4d9d-a417-55816c14c8bd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:05:38.059867 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ad0da2-90c4-4d9d-a417-55816c14c8bd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:38.061255 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ad0da2-90c4-4d9d-a417-55816c14c8bd None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:38.074328 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 491/966] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Jul 14 09:05:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:05:38.079777 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380db722-03af-42da-b732-f883e508d691 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:38.141804 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:38.183586 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380db722-03af-42da-b732-f883e508d691 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:38.222061 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0404a831-0344-4afe-9162-9de13f6f14f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:38.223178 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0404a831-0344-4afe-9162-9de13f6f14f2 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:38.223633 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0404a831-0344-4afe-9162-9de13f6f14f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:38.224042 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0404a831-0344-4afe-9162-9de13f6f14f2 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:38.232579 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0404a831-0344-4afe-9162-9de13f6f14f2 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:38.234275 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0404a831-0344-4afe-9162-9de13f6f14f2 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:38.249469 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-0404a831-0344-4afe-9162-9de13f6f14f2 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 14 09:05:38.249469 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:05:38.249469 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:05:38.249469 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:05:38.249469 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:05:38.249469 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:05:38.249469 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:05:38.249469 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:05:38.249469 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:05:38.249469 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:05:38.249469 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:05:38.249469 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:05:38.249469 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 09:05:38.249469 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 09:05:38.249469 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 09:05:38.249469 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 09:05:38.250693 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:05:38.250693 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:05:38.250693 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 09:05:38.250693 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 09:05:38.250693 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 09:05:38.250693 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 09:05:38.250693 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:05:38.250693 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:05:38.250693 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:05:38.250693 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:05:38.250693 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:05:38.250693 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 14 09:05:38.250693 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:05:38.250693 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:05:38.250693 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:05:38.250693 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:05:38.250693 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:05:38.250693 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:05:38.251944 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:05:38.251944 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:05:38.251944 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 09:05:38.251944 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 09:05:38.251944 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 09:05:38.251944 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 09:05:38.251944 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 09:05:38.251944 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 09:05:38.251944 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 14 09:05:38.251944 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 14 09:05:38.254289 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 476/967] 10.222.0.26 () {56 vars in 1117 bytes} [Thu Jul 14 09:05:38 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:05:38.260186 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ad0da2-90c4-4d9d-a417-55816c14c8bd None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:38.360806 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ad0da2-90c4-4d9d-a417-55816c14c8bd None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:38.381919 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8c4a9f-301f-4199-adf9-1df40568fcc3 None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:38.382204 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8c4a9f-301f-4199-adf9-1df40568fcc3 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:38.382657 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8c4a9f-301f-4199-adf9-1df40568fcc3 None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:38.384331 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 477/968] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:05:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 09:05:38.389164 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6e4194-5d4a-4e71-883e-27a3d4143089 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:38.389447 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6e4194-5d4a-4e71-883e-27a3d4143089 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:38.389719 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6e4194-5d4a-4e71-883e-27a3d4143089 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:38.398506 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:38.410709 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-4a6e4194-5d4a-4e71-883e-27a3d4143089 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:05:38.411061 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fa3891-2ce8-4dd4-a25e-a1698d5fc825 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:38.411779 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fa3891-2ce8-4dd4-a25e-a1698d5fc825 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:38.412066 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fa3891-2ce8-4dd4-a25e-a1698d5fc825 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:38.412342 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fa3891-2ce8-4dd4-a25e-a1698d5fc825 None admin] PATH_INFO: `/v3/users` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:38.412963 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-4a6e4194-5d4a-4e71-883e-27a3d4143089 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:05:38.413193 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fa3891-2ce8-4dd4-a25e-a1698d5fc825 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:05:38.413504 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fa3891-2ce8-4dd4-a25e-a1698d5fc825 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:38.414943 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fa3891-2ce8-4dd4-a25e-a1698d5fc825 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:38.433462 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 492/969] 10.222.0.26 () {56 vars in 1149 bytes} [Thu Jul 14 09:05:38 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:05:38.440380 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fa3891-2ce8-4dd4-a25e-a1698d5fc825 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:38.497058 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 478/970] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:05:38 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 09:05:38.521169 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6e4194-5d4a-4e71-883e-27a3d4143089 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:38.565073 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fa3891-2ce8-4dd4-a25e-a1698d5fc825 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:38.587822 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:38.599993 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e52fa3d-4f2b-4667-9dfe-9474924bf1c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:38.600658 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e52fa3d-4f2b-4667-9dfe-9474924bf1c6 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:38.600951 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e52fa3d-4f2b-4667-9dfe-9474924bf1c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:38.601222 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e52fa3d-4f2b-4667-9dfe-9474924bf1c6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:38.641736 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6e4194-5d4a-4e71-883e-27a3d4143089 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:38.659265 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:38.669361 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e52fa3d-4f2b-4667-9dfe-9474924bf1c6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:38.670832 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e52fa3d-4f2b-4667-9dfe-9474924bf1c6 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:38.682022 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacb94a1-62f9-4965-adcd-7b72ef41d8bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:38.683122 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacb94a1-62f9-4965-adcd-7b72ef41d8bd None admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:38.683639 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacb94a1-62f9-4965-adcd-7b72ef41d8bd None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:38.684059 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacb94a1-62f9-4965-adcd-7b72ef41d8bd None admin] PATH_INFO: `/v3/users` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:38.684939 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacb94a1-62f9-4965-adcd-7b72ef41d8bd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:38.686671 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacb94a1-62f9-4965-adcd-7b72ef41d8bd None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:38.688266 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: WARNING keystone.server.flask.common [None req-eacb94a1-62f9-4965-adcd-7b72ef41d8bd None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 14 09:05:38.702758 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.sql.core [None req-eacb94a1-62f9-4965-adcd-7b72ef41d8bd None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 14 09:05:38.702758 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 14 09:05:38.702758 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [parameters: {'user_id': 'b379b672ddd54d48ab8d3c28339915d2', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 14 09:05:38.702758 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80381) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 14 09:05:38.711794 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-eacb94a1-62f9-4965-adcd-7b72ef41d8bd None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 14 09:05:38.711794 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:05:38.711794 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 14 09:05:38.711794 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 14 09:05:38.711794 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 14 09:05:38.711794 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 14 09:05:38.711794 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 14 09:05:38.711794 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application result = self._query(query) Jul 14 09:05:38.711794 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 14 09:05:38.711794 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application conn.query(q) Jul 14 09:05:38.711794 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 14 09:05:38.711794 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 14 09:05:38.711794 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 14 09:05:38.711794 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application result.read() Jul 14 09:05:38.711794 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 14 09:05:38.713052 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 14 09:05:38.713052 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 14 09:05:38.713052 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 14 09:05:38.713052 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 14 09:05:38.713052 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 14 09:05:38.713052 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 14 09:05:38.713052 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 14 09:05:38.713052 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 14 09:05:38.713052 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 14 09:05:38.713052 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 14 09:05:38.713052 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 14 09:05:38.713052 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:05:38.713052 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 14 09:05:38.713052 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 14 09:05:38.713052 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jul 14 09:05:38.713052 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jul 14 09:05:38.713052 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 14 09:05:38.713052 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application next(self.gen) Jul 14 09:05:38.714178 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 14 09:05:38.714178 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application yield resource Jul 14 09:05:38.714178 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 14 09:05:38.714178 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application next(self.gen) Jul 14 09:05:38.714178 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 14 09:05:38.714178 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application self.session.rollback() Jul 14 09:05:38.714178 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 14 09:05:38.714178 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application self.force_reraise() Jul 14 09:05:38.714178 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 14 09:05:38.714178 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise self.value Jul 14 09:05:38.714178 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 14 09:05:38.714178 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 14 09:05:38.714178 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 14 09:05:38.714178 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application session.commit() Jul 14 09:05:38.714178 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 14 09:05:38.714178 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 14 09:05:38.714178 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 14 09:05:38.715134 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application self._prepare_impl() Jul 14 09:05:38.715134 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 14 09:05:38.715134 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application self.session.flush() Jul 14 09:05:38.715134 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 14 09:05:38.715134 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application self._flush(objects) Jul 14 09:05:38.715134 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 14 09:05:38.715134 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 14 09:05:38.715134 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 14 09:05:38.715134 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application compat.raise_( Jul 14 09:05:38.715134 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 14 09:05:38.715134 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception Jul 14 09:05:38.715134 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 14 09:05:38.715134 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application flush_context.execute() Jul 14 09:05:38.715134 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 14 09:05:38.715134 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rec.execute(self) Jul 14 09:05:38.715134 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 14 09:05:38.715134 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 14 09:05:38.716276 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 14 09:05:38.716276 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 14 09:05:38.716276 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jul 14 09:05:38.716276 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application result = connection._execute_20( Jul 14 09:05:38.716276 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 14 09:05:38.716276 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 14 09:05:38.716276 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 14 09:05:38.716276 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 14 09:05:38.716276 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 14 09:05:38.716276 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 14 09:05:38.716276 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 14 09:05:38.716276 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 14 09:05:38.716276 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 14 09:05:38.716276 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 14 09:05:38.716276 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 14 09:05:38.716276 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception Jul 14 09:05:38.717220 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 14 09:05:38.717220 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 14 09:05:38.717220 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 14 09:05:38.717220 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 14 09:05:38.717220 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 14 09:05:38.717220 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application result = self._query(query) Jul 14 09:05:38.717220 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 14 09:05:38.717220 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application conn.query(q) Jul 14 09:05:38.717220 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 14 09:05:38.717220 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 14 09:05:38.717220 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 14 09:05:38.717220 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application result.read() Jul 14 09:05:38.717220 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 14 09:05:38.717220 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 14 09:05:38.717220 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 14 09:05:38.717220 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 14 09:05:38.718131 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 14 09:05:38.718131 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 14 09:05:38.718131 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 14 09:05:38.718131 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 14 09:05:38.718131 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 14 09:05:38.718131 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 14 09:05:38.718131 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [parameters: {'user_id': 'b379b672ddd54d48ab8d3c28339915d2', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 14 09:05:38.718131 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 14 09:05:38.718131 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 14 09:05:38.718131 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 09:05:38.718131 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 14 09:05:38.718131 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:05:38.718131 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:05:38.718131 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:05:38.718131 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:05:38.719043 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:05:38.719043 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:05:38.719043 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:05:38.719043 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:05:38.719043 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:05:38.719043 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:05:38.719043 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:05:38.719043 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jul 14 09:05:38.719043 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jul 14 09:05:38.719043 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:05:38.719043 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:05:38.719043 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 09:05:38.719043 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 09:05:38.719043 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 09:05:38.719043 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 09:05:38.719043 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jul 14 09:05:38.719043 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jul 14 09:05:38.720083 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jul 14 09:05:38.720083 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jul 14 09:05:38.720083 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 14 09:05:38.720083 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 14 09:05:38.720083 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 14 09:05:38.720083 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 14 09:05:38.727316 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 479/971] 10.222.0.26 () {60 vars in 1131 bytes} [Thu Jul 14 09:05:38 2022] POST /identity/v3/users => generated 160 bytes in 202 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:05:38.729149 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacb94a1-62f9-4965-adcd-7b72ef41d8bd None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:38.774222 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 493/972] 10.222.0.26 () {58 vars in 1282 bytes} [Thu Jul 14 09:05:38 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 09:05:38.779454 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e52fa3d-4f2b-4667-9dfe-9474924bf1c6 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:38.841590 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacb94a1-62f9-4965-adcd-7b72ef41d8bd None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:38.857178 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:38.879014 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:38.882171 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c5d658-a89b-4c03-acf8-5f0b4256e7bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:38.883143 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c5d658-a89b-4c03-acf8-5f0b4256e7bb None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:38.883686 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c5d658-a89b-4c03-acf8-5f0b4256e7bb None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:38.884187 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c5d658-a89b-4c03-acf8-5f0b4256e7bb None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:38.889789 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e52fa3d-4f2b-4667-9dfe-9474924bf1c6 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:38.893438 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c5d658-a89b-4c03-acf8-5f0b4256e7bb None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:38.895342 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c5d658-a89b-4c03-acf8-5f0b4256e7bb None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:38.911740 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-49c5d658-a89b-4c03-acf8-5f0b4256e7bb None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 14 09:05:38.911740 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:05:38.911740 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:05:38.911740 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:05:38.911740 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:05:38.911740 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:05:38.911740 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:05:38.911740 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:05:38.911740 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:05:38.911740 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:05:38.911740 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:05:38.911740 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:05:38.911740 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 09:05:38.911740 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 09:05:38.911740 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 09:05:38.911740 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 09:05:38.912717 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:05:38.912717 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:05:38.912717 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 09:05:38.912717 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 09:05:38.912717 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 09:05:38.912717 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 09:05:38.912717 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:05:38.912717 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:05:38.912717 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:05:38.912717 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:05:38.912717 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:05:38.912717 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 14 09:05:38.912717 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:05:38.912717 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:05:38.912717 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:05:38.912717 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:05:38.912717 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:05:38.912717 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:05:38.913686 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:05:38.913686 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:05:38.913686 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 09:05:38.913686 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 09:05:38.913686 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 09:05:38.913686 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 09:05:38.913686 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 09:05:38.913686 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 09:05:38.913686 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 14 09:05:38.913686 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 14 09:05:38.920921 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 480/973] 10.222.0.26 () {56 vars in 1099 bytes} [Thu Jul 14 09:05:38 2022] GET /identity/v3/users/cinder => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:05:38.922149 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c5d658-a89b-4c03-acf8-5f0b4256e7bb None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:38.926509 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8863045a-66ad-45a8-8388-80023c219120 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:38.926656 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8863045a-66ad-45a8-8388-80023c219120 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:38.926937 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8863045a-66ad-45a8-8388-80023c219120 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:38.927235 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8863045a-66ad-45a8-8388-80023c219120 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:38.938288 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8863045a-66ad-45a8-8388-80023c219120 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:38.939899 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8863045a-66ad-45a8-8388-80023c219120 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:38.959768 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-8863045a-66ad-45a8-8388-80023c219120 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 09:05:38.959768 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:05:38.959768 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:05:38.959768 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:05:38.959768 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:05:38.959768 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:05:38.959768 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:05:38.959768 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:05:38.959768 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:05:38.959768 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:05:38.959768 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:05:38.959768 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:05:38.959768 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 09:05:38.959768 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 09:05:38.959768 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 09:05:38.960740 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 09:05:38.960740 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:05:38.960740 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:05:38.960740 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:05:38.960740 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:05:38.960740 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:05:38.960740 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:05:38.960740 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:05:38.960740 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 14 09:05:38.960740 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:05:38.960740 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:05:38.960740 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:05:38.960740 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:05:38.960740 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:05:38.960740 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:05:38.960740 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:05:38.960740 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:05:38.961669 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 09:05:38.961669 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 09:05:38.961669 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 09:05:38.961669 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 14 09:05:38.961669 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 494/974] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jul 14 09:05:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 09:05:38.966779 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8863045a-66ad-45a8-8388-80023c219120 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:39.033587 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c5d658-a89b-4c03-acf8-5f0b4256e7bb None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:39.071586 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb59b28-ab30-4afa-be37-c913b9c71d3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:39.072710 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb59b28-ab30-4afa-be37-c913b9c71d3c None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:39.073204 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb59b28-ab30-4afa-be37-c913b9c71d3c None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:39.073723 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb59b28-ab30-4afa-be37-c913b9c71d3c None admin] PATH_INFO: `/v3/users` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:39.074874 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb59b28-ab30-4afa-be37-c913b9c71d3c None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:05:39.075435 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb59b28-ab30-4afa-be37-c913b9c71d3c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:39.077139 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb59b28-ab30-4afa-be37-c913b9c71d3c None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:39.091050 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8863045a-66ad-45a8-8388-80023c219120 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:39.127636 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 481/975] 10.222.0.26 () {56 vars in 1101 bytes} [Thu Jul 14 09:05:38 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:05:39.129629 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67678c3e-14d9-4c3b-b3e4-5d42cf90faa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:39.130305 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67678c3e-14d9-4c3b-b3e4-5d42cf90faa4 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:39.130594 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67678c3e-14d9-4c3b-b3e4-5d42cf90faa4 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:39.130866 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67678c3e-14d9-4c3b-b3e4-5d42cf90faa4 None admin] PATH_INFO: `/v3/domains` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:39.131652 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67678c3e-14d9-4c3b-b3e4-5d42cf90faa4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:05:39.131963 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67678c3e-14d9-4c3b-b3e4-5d42cf90faa4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:39.133374 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67678c3e-14d9-4c3b-b3e4-5d42cf90faa4 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:39.148135 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 495/976] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Jul 14 09:05:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:05:39.154086 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb59b28-ab30-4afa-be37-c913b9c71d3c None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:39.197214 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:39.273887 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb59b28-ab30-4afa-be37-c913b9c71d3c None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:39.310640 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953f5ada-afa6-4703-a67d-8d3ea2c046b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:39.311403 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953f5ada-afa6-4703-a67d-8d3ea2c046b7 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:39.311689 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953f5ada-afa6-4703-a67d-8d3ea2c046b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:39.311962 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953f5ada-afa6-4703-a67d-8d3ea2c046b7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:39.319426 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953f5ada-afa6-4703-a67d-8d3ea2c046b7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:39.320885 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953f5ada-afa6-4703-a67d-8d3ea2c046b7 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:39.335150 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-953f5ada-afa6-4703-a67d-8d3ea2c046b7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 09:05:39.335150 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:05:39.335150 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:05:39.335150 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:05:39.335150 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:05:39.335150 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:05:39.335150 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:05:39.335150 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:05:39.335150 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:05:39.335150 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:05:39.335150 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:05:39.335150 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:05:39.335150 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 09:05:39.335150 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 09:05:39.335150 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 09:05:39.336125 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 09:05:39.336125 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:05:39.336125 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:05:39.336125 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:05:39.336125 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:05:39.336125 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:05:39.336125 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:05:39.336125 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:05:39.336125 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 14 09:05:39.336125 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:05:39.336125 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:05:39.336125 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:05:39.336125 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:05:39.336125 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:05:39.336125 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:05:39.336125 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:05:39.336125 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:05:39.337049 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 09:05:39.337049 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 09:05:39.337049 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 09:05:39.337049 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 09:05:39.337049 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 09:05:39.337049 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 09:05:39.337049 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 09:05:39.337049 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 14 09:05:39.337049 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 482/977] 10.222.0.26 () {56 vars in 1111 bytes} [Thu Jul 14 09:05:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:05:39.342130 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67678c3e-14d9-4c3b-b3e4-5d42cf90faa4 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:39.345207 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:39.422773 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc20fa2-84b9-46db-a008-0192b589e04d None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:39.423071 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc20fa2-84b9-46db-a008-0192b589e04d None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:39.423517 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc20fa2-84b9-46db-a008-0192b589e04d None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:39.425101 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 483/978] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:05:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 09:05:39.429927 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967c068b-2d49-48fc-915e-01e9425b079b None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:39.430346 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967c068b-2d49-48fc-915e-01e9425b079b None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:39.430688 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967c068b-2d49-48fc-915e-01e9425b079b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:39.446461 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67678c3e-14d9-4c3b-b3e4-5d42cf90faa4 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:39.449163 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-967c068b-2d49-48fc-915e-01e9425b079b None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:05:39.450668 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-967c068b-2d49-48fc-915e-01e9425b079b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:05:39.480977 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc0c8ab-9945-4374-a998-8fbc2ca5a10e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:39.481644 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc0c8ab-9945-4374-a998-8fbc2ca5a10e None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:39.481927 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc0c8ab-9945-4374-a998-8fbc2ca5a10e None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:39.482202 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc0c8ab-9945-4374-a998-8fbc2ca5a10e None admin] PATH_INFO: `/v3/projects` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:39.483002 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc0c8ab-9945-4374-a998-8fbc2ca5a10e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:05:39.483358 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc0c8ab-9945-4374-a998-8fbc2ca5a10e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:39.484854 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc0c8ab-9945-4374-a998-8fbc2ca5a10e None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:39.502201 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 496/979] 10.222.0.26 () {56 vars in 1148 bytes} [Thu Jul 14 09:05:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 09:05:39.503348 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc0c8ab-9945-4374-a998-8fbc2ca5a10e None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:39.526758 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 484/980] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:05:39 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 09:05:39.553479 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967c068b-2d49-48fc-915e-01e9425b079b None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:39.578829 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:39.579908 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:39.621883 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc0c8ab-9945-4374-a998-8fbc2ca5a10e None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:39.657322 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49aa00d7-8ae7-47f3-a0cf-1be6665c00ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:39.657729 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967c068b-2d49-48fc-915e-01e9425b079b None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:39.657978 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49aa00d7-8ae7-47f3-a0cf-1be6665c00ac None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:39.658264 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49aa00d7-8ae7-47f3-a0cf-1be6665c00ac None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:39.658540 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49aa00d7-8ae7-47f3-a0cf-1be6665c00ac None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:39.659407 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49aa00d7-8ae7-47f3-a0cf-1be6665c00ac None admin] RBAC: Adding query filter params (role.id=70488f05098d4ce09e2fb03aaaea4566, scope.project.id=33bf210d015847439655b9943d1b0261, user.id=7d6267ede8d64e2abe1f4058a9797fb7) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:05:39.659726 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49aa00d7-8ae7-47f3-a0cf-1be6665c00ac None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:39.661154 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49aa00d7-8ae7-47f3-a0cf-1be6665c00ac None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:39.667078 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 497/981] 10.222.0.26 () {56 vars in 1374 bytes} [Thu Jul 14 09:05:39 2022] GET /identity/v3/role_assignments?user.id=7d6267ede8d64e2abe1f4058a9797fb7&scope.project.id=33bf210d015847439655b9943d1b0261&role.id=70488f05098d4ce09e2fb03aaaea4566 => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 09:05:39.668134 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e16625-ea7c-4a6d-924b-0ad46a828cfe None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:39.668425 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e16625-ea7c-4a6d-924b-0ad46a828cfe None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:39.668705 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e16625-ea7c-4a6d-924b-0ad46a828cfe None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:39.670055 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 498/982] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:05:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:05:39.675809 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749548e7-a3e6-43b6-8b44-8f54fd24369d None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:39.676125 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749548e7-a3e6-43b6-8b44-8f54fd24369d None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:39.676413 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749548e7-a3e6-43b6-8b44-8f54fd24369d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:39.693830 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-749548e7-a3e6-43b6-8b44-8f54fd24369d None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:05:39.695089 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-749548e7-a3e6-43b6-8b44-8f54fd24369d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:05:39.695782 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ac1322-f0ab-43c4-9de3-9c429876a41d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:39.696618 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ac1322-f0ab-43c4-9de3-9c429876a41d None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:39.696900 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ac1322-f0ab-43c4-9de3-9c429876a41d None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:39.697364 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ac1322-f0ab-43c4-9de3-9c429876a41d None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:39.697842 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ac1322-f0ab-43c4-9de3-9c429876a41d None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:39.699278 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ac1322-f0ab-43c4-9de3-9c429876a41d None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:39.712231 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-d3ac1322-f0ab-43c4-9de3-9c429876a41d None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 14 09:05:39.712231 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:05:39.712231 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 09:05:39.712231 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 09:05:39.712231 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 09:05:39.712231 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 09:05:39.712231 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 09:05:39.712231 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 09:05:39.712231 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 14 09:05:39.712231 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 14 09:05:39.712231 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 09:05:39.712231 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 14 09:05:39.712231 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:05:39.712231 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:05:39.712231 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:05:39.712231 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:05:39.712231 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:05:39.713364 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:05:39.713364 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:05:39.713364 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:05:39.713364 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:05:39.713364 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:05:39.713364 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:05:39.713364 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 09:05:39.713364 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 09:05:39.713364 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 09:05:39.713364 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 09:05:39.713364 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:05:39.713364 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:05:39.713364 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:05:39.713364 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:05:39.713364 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:05:39.713364 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:05:39.714227 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:05:39.714227 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 14 09:05:39.714227 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:05:39.714227 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:05:39.714227 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:05:39.714227 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:05:39.714227 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:05:39.714227 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:05:39.714227 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:05:39.714227 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:05:39.714227 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 09:05:39.714227 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 09:05:39.714227 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 14 09:05:39.714227 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 14 09:05:39.714227 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 485/983] 10.222.0.26 () {56 vars in 1126 bytes} [Thu Jul 14 09:05:39 2022] GET /identity/v3/services/object-store => generated 93 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:05:39.719838 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ac1322-f0ab-43c4-9de3-9c429876a41d None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:39.782448 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 499/984] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:05:39 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 09:05:39.805982 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749548e7-a3e6-43b6-8b44-8f54fd24369d None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:39.825864 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:39.848285 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ac1322-f0ab-43c4-9de3-9c429876a41d None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:39.878188 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:39.886102 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de824be-f727-440c-8846-0928f5be67d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:39.886773 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de824be-f727-440c-8846-0928f5be67d7 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:39.887060 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de824be-f727-440c-8846-0928f5be67d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:39.887371 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de824be-f727-440c-8846-0928f5be67d7 None admin] PATH_INFO: `/v3/services` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:39.888084 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de824be-f727-440c-8846-0928f5be67d7 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:05:39.888390 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de824be-f727-440c-8846-0928f5be67d7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:39.889766 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de824be-f727-440c-8846-0928f5be67d7 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:39.895062 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 486/985] 10.222.0.26 () {56 vars in 1122 bytes} [Thu Jul 14 09:05:39 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 09:05:39.900362 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de824be-f727-440c-8846-0928f5be67d7 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:39.929628 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749548e7-a3e6-43b6-8b44-8f54fd24369d None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:39.966667 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfcd03e-794a-4a8c-887a-cde0adcee7e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:39.967387 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfcd03e-794a-4a8c-887a-cde0adcee7e1 None admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:39.967651 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfcd03e-794a-4a8c-887a-cde0adcee7e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:39.967932 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfcd03e-794a-4a8c-887a-cde0adcee7e1 None admin] PATH_INFO: `/v3/services` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:39.968782 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfcd03e-794a-4a8c-887a-cde0adcee7e1 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:39.970155 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfcd03e-794a-4a8c-887a-cde0adcee7e1 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:39.985660 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 500/986] 10.222.0.26 () {60 vars in 1141 bytes} [Thu Jul 14 09:05:39 2022] POST /identity/v3/services => generated 242 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 09:05:40.013725 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de824be-f727-440c-8846-0928f5be67d7 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:40.060637 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3142532-efa9-4e70-b833-65d22a5e74c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:40.061320 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3142532-efa9-4e70-b833-65d22a5e74c5 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:40.061603 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3142532-efa9-4e70-b833-65d22a5e74c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:40.061875 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3142532-efa9-4e70-b833-65d22a5e74c5 None admin] PATH_INFO: `/v3/services` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:40.062669 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3142532-efa9-4e70-b833-65d22a5e74c5 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:05:40.062973 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3142532-efa9-4e70-b833-65d22a5e74c5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:40.064421 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3142532-efa9-4e70-b833-65d22a5e74c5 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:40.070503 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 487/987] 10.222.0.26 () {56 vars in 1122 bytes} [Thu Jul 14 09:05:39 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 09:05:40.076196 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfcd03e-794a-4a8c-887a-cde0adcee7e1 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:40.191231 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfcd03e-794a-4a8c-887a-cde0adcee7e1 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:40.209261 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:40.231621 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab7cd3a-8c41-411f-80ba-c57960eea76f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:40.232305 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab7cd3a-8c41-411f-80ba-c57960eea76f None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:40.232583 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab7cd3a-8c41-411f-80ba-c57960eea76f None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:40.232861 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab7cd3a-8c41-411f-80ba-c57960eea76f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:40.233672 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab7cd3a-8c41-411f-80ba-c57960eea76f None admin] RBAC: Adding query filter params (interface=admin, service_id=ff4db0acaf2c4a219bdfe8c3341df62d, region_id=RegionOne) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:05:40.233996 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab7cd3a-8c41-411f-80ba-c57960eea76f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:40.235437 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab7cd3a-8c41-411f-80ba-c57960eea76f None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:40.241058 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 501/988] 10.222.0.26 () {56 vars in 1249 bytes} [Thu Jul 14 09:05:40 2022] GET /identity/v3/endpoints?service_id=ff4db0acaf2c4a219bdfe8c3341df62d&interface=admin®ion_id=RegionOne => generated 193 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:05:40.246699 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3142532-efa9-4e70-b833-65d22a5e74c5 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:40.259951 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:40.352841 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3142532-efa9-4e70-b833-65d22a5e74c5 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:40.392338 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56927c20-1502-46ed-9733-017bc178b803 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:40.393015 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56927c20-1502-46ed-9733-017bc178b803 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:40.393303 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56927c20-1502-46ed-9733-017bc178b803 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:40.393580 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56927c20-1502-46ed-9733-017bc178b803 None admin] PATH_INFO: `/v3/services` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:40.394270 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56927c20-1502-46ed-9733-017bc178b803 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:40.395700 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56927c20-1502-46ed-9733-017bc178b803 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:40.402659 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 488/989] 10.222.0.26 () {56 vars in 1087 bytes} [Thu Jul 14 09:05:40 2022] GET /identity/v3/services => generated 1727 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 09:05:41.711132 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e78e32-2cc6-4fb6-84db-065727909fa5 None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:41.711773 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e78e32-2cc6-4fb6-84db-065727909fa5 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:41.711773 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e78e32-2cc6-4fb6-84db-065727909fa5 None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:41.713077 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 502/990] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:05:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:05:41.718556 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2443261d-d905-4148-8eca-fea000c683e1 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:41.718974 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2443261d-d905-4148-8eca-fea000c683e1 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:41.719111 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2443261d-d905-4148-8eca-fea000c683e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:41.736598 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-2443261d-d905-4148-8eca-fea000c683e1 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:05:41.737782 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-2443261d-d905-4148-8eca-fea000c683e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:05:41.814632 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 489/991] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:05:41 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 09:05:41.831575 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e78e32-2cc6-4fb6-84db-065727909fa5 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:41.942306 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e78e32-2cc6-4fb6-84db-065727909fa5 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:41.976340 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ffa3cf-d7cf-4bd6-8199-04536e8b8034 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:41.977142 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ffa3cf-d7cf-4bd6-8199-04536e8b8034 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:41.977420 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ffa3cf-d7cf-4bd6-8199-04536e8b8034 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:41.977693 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ffa3cf-d7cf-4bd6-8199-04536e8b8034 None admin] PATH_INFO: `/v3/services/image` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:41.978352 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ffa3cf-d7cf-4bd6-8199-04536e8b8034 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:41.979764 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ffa3cf-d7cf-4bd6-8199-04536e8b8034 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:41.991635 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-57ffa3cf-d7cf-4bd6-8199-04536e8b8034 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 14 09:05:41.991635 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:05:41.991635 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 09:05:41.991635 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 09:05:41.991635 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 09:05:41.991635 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 09:05:41.991635 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 09:05:41.991635 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 09:05:41.991635 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 14 09:05:41.991635 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 14 09:05:41.991635 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 09:05:41.991635 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 14 09:05:41.991635 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:05:41.991635 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:05:41.991635 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:05:41.991635 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:05:41.991635 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:05:41.992642 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:05:41.992642 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:05:41.992642 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:05:41.992642 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:05:41.992642 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:05:41.992642 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:05:41.992642 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 09:05:41.992642 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 09:05:41.992642 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 09:05:41.992642 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 09:05:41.992642 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:05:41.992642 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:05:41.992642 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:05:41.992642 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:05:41.992642 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:05:41.992642 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:05:41.998339 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:05:41.998339 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 14 09:05:41.998339 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:05:41.998339 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:05:41.998339 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:05:41.998339 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:05:41.998339 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:05:41.998339 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:05:41.998339 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:05:41.998339 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:05:41.998339 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 09:05:41.998339 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 09:05:41.998339 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 14 09:05:41.998339 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 14 09:05:41.998339 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 503/992] 10.222.0.26 () {56 vars in 1105 bytes} [Thu Jul 14 09:05:41 2022] GET /identity/v3/services/image => generated 86 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:05:41.999156 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2443261d-d905-4148-8eca-fea000c683e1 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:42.002238 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:42.013245 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa7c097-7eb9-44df-94c6-59c65f5ecc94 None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:42.013530 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa7c097-7eb9-44df-94c6-59c65f5ecc94 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:42.013804 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa7c097-7eb9-44df-94c6-59c65f5ecc94 None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:42.015423 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 504/993] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:05:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:05:42.020421 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33f69bf-a34b-418e-a66a-4524b6e4cad9 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:42.020703 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33f69bf-a34b-418e-a66a-4524b6e4cad9 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:42.020975 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33f69bf-a34b-418e-a66a-4524b6e4cad9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:42.038705 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-a33f69bf-a34b-418e-a66a-4524b6e4cad9 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:05:42.039923 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-a33f69bf-a34b-418e-a66a-4524b6e4cad9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:05:42.082599 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:42.110113 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2443261d-d905-4148-8eca-fea000c683e1 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:42.112168 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 505/994] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:05:42 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 09:05:42.134535 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33f69bf-a34b-418e-a66a-4524b6e4cad9 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:42.151697 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dd1ef9-d964-44fe-96af-dd3ffff2a0ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:42.152921 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dd1ef9-d964-44fe-96af-dd3ffff2a0ad None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:42.153490 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dd1ef9-d964-44fe-96af-dd3ffff2a0ad None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:42.153959 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dd1ef9-d964-44fe-96af-dd3ffff2a0ad None admin] PATH_INFO: `/v3/services` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:42.155278 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25dd1ef9-d964-44fe-96af-dd3ffff2a0ad None admin] RBAC: Adding query filter params (name=image) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:05:42.156187 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25dd1ef9-d964-44fe-96af-dd3ffff2a0ad None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:42.158293 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25dd1ef9-d964-44fe-96af-dd3ffff2a0ad None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:42.165742 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 490/995] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jul 14 09:05:41 2022] GET /identity/v3/services?name=image => generated 122 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 09:05:42.170950 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dd1ef9-d964-44fe-96af-dd3ffff2a0ad None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:42.238408 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33f69bf-a34b-418e-a66a-4524b6e4cad9 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:42.239543 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:42.276655 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dd1ef9-d964-44fe-96af-dd3ffff2a0ad None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:42.281772 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8275c3c2-70e2-448c-bf9d-585c0d239783 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:42.282588 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8275c3c2-70e2-448c-bf9d-585c0d239783 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:42.282925 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8275c3c2-70e2-448c-bf9d-585c0d239783 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:42.283225 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8275c3c2-70e2-448c-bf9d-585c0d239783 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:42.283936 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8275c3c2-70e2-448c-bf9d-585c0d239783 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:42.285343 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8275c3c2-70e2-448c-bf9d-585c0d239783 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:42.300397 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-8275c3c2-70e2-448c-bf9d-585c0d239783 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 14 09:05:42.300397 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:05:42.300397 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 09:05:42.300397 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 09:05:42.300397 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 09:05:42.300397 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 09:05:42.300397 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 09:05:42.300397 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 09:05:42.300397 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 14 09:05:42.300397 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 14 09:05:42.300397 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 09:05:42.300397 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 14 09:05:42.300397 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:05:42.300397 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:05:42.300397 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:05:42.300397 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:05:42.300397 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:05:42.301456 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:05:42.301456 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:05:42.301456 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:05:42.301456 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:05:42.301456 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:05:42.301456 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:05:42.301456 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 09:05:42.301456 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 09:05:42.301456 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 09:05:42.301456 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 09:05:42.301456 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:05:42.301456 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:05:42.301456 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:05:42.301456 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:05:42.301456 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:05:42.301456 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:05:42.302313 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:05:42.302313 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 14 09:05:42.302313 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:05:42.302313 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:05:42.302313 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:05:42.302313 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:05:42.302313 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:05:42.302313 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:05:42.302313 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:05:42.302313 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:05:42.302313 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 09:05:42.302313 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 09:05:42.302313 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 14 09:05:42.302313 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 14 09:05:42.307122 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 506/996] 10.222.0.26 () {56 vars in 1111 bytes} [Thu Jul 14 09:05:42 2022] GET /identity/v3/services/compute => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 09:05:42.308281 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8275c3c2-70e2-448c-bf9d-585c0d239783 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:42.318465 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ca6ee5-e8d7-46af-8cdb-684f4c558627 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:42.319133 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ca6ee5-e8d7-46af-8cdb-684f4c558627 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:42.319445 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ca6ee5-e8d7-46af-8cdb-684f4c558627 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:42.319721 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ca6ee5-e8d7-46af-8cdb-684f4c558627 None admin] PATH_INFO: `/v3/services` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:42.320424 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ca6ee5-e8d7-46af-8cdb-684f4c558627 None admin] RBAC: Adding query filter params (type=image) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:05:42.320726 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ca6ee5-e8d7-46af-8cdb-684f4c558627 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:42.322098 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ca6ee5-e8d7-46af-8cdb-684f4c558627 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:42.326908 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 491/997] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jul 14 09:05:42 2022] GET /identity/v3/services?type=image => generated 122 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 09:05:42.355649 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:42.421139 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc160c98-772e-4bf5-955b-2b3c20f35cfd None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:42.421494 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc160c98-772e-4bf5-955b-2b3c20f35cfd None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:42.422031 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc160c98-772e-4bf5-955b-2b3c20f35cfd None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:42.423235 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 492/998] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:05:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:05:42.429010 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc76521a-bb0d-4927-8d47-cf3ef4ee867f None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:42.429330 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc76521a-bb0d-4927-8d47-cf3ef4ee867f None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:42.429685 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc76521a-bb0d-4927-8d47-cf3ef4ee867f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:42.433110 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8275c3c2-70e2-448c-bf9d-585c0d239783 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:42.447925 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-fc76521a-bb0d-4927-8d47-cf3ef4ee867f None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:05:42.449160 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-fc76521a-bb0d-4927-8d47-cf3ef4ee867f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:05:42.475952 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663bec2d-cd73-4b8f-9db2-c1878ca33b38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:42.476607 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663bec2d-cd73-4b8f-9db2-c1878ca33b38 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:42.476892 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663bec2d-cd73-4b8f-9db2-c1878ca33b38 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:42.477169 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663bec2d-cd73-4b8f-9db2-c1878ca33b38 None admin] PATH_INFO: `/v3/services` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:42.477886 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663bec2d-cd73-4b8f-9db2-c1878ca33b38 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:05:42.478192 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663bec2d-cd73-4b8f-9db2-c1878ca33b38 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:42.479640 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663bec2d-cd73-4b8f-9db2-c1878ca33b38 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:42.485148 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 507/999] 10.222.0.26 () {56 vars in 1112 bytes} [Thu Jul 14 09:05:42 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:05:42.490314 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663bec2d-cd73-4b8f-9db2-c1878ca33b38 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:42.525952 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 493/1000] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:05:42 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 09:05:42.552049 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc76521a-bb0d-4927-8d47-cf3ef4ee867f None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:42.605153 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663bec2d-cd73-4b8f-9db2-c1878ca33b38 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:42.607000 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:42.619835 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:42.642459 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b741b3b-7d95-4bab-b775-069338a8e29e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:42.643154 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b741b3b-7d95-4bab-b775-069338a8e29e None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:42.643449 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b741b3b-7d95-4bab-b775-069338a8e29e None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:42.643728 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b741b3b-7d95-4bab-b775-069338a8e29e None admin] PATH_INFO: `/v3/services` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:42.644445 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b741b3b-7d95-4bab-b775-069338a8e29e None admin] RBAC: Adding query filter params (type=compute) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:05:42.644753 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b741b3b-7d95-4bab-b775-069338a8e29e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:42.646128 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b741b3b-7d95-4bab-b775-069338a8e29e None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:42.651164 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 508/1001] 10.222.0.26 () {56 vars in 1112 bytes} [Thu Jul 14 09:05:42 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:05:42.657229 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b741b3b-7d95-4bab-b775-069338a8e29e None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:42.670510 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc76521a-bb0d-4927-8d47-cf3ef4ee867f None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:42.707087 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e9d59a-d1a4-4623-a820-6237bebd699e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:42.707927 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e9d59a-d1a4-4623-a820-6237bebd699e None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:42.708213 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e9d59a-d1a4-4623-a820-6237bebd699e None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:42.708491 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e9d59a-d1a4-4623-a820-6237bebd699e None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:42.709147 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e9d59a-d1a4-4623-a820-6237bebd699e None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:42.710522 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e9d59a-d1a4-4623-a820-6237bebd699e None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:42.722803 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-c2e9d59a-d1a4-4623-a820-6237bebd699e None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 14 09:05:42.722803 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:05:42.722803 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 09:05:42.722803 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 09:05:42.722803 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 09:05:42.722803 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 09:05:42.722803 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 09:05:42.722803 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 09:05:42.722803 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 14 09:05:42.722803 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 14 09:05:42.722803 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 09:05:42.722803 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 14 09:05:42.722803 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:05:42.722803 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:05:42.722803 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:05:42.722803 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:05:42.722803 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:05:42.724287 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:05:42.724287 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:05:42.724287 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:05:42.724287 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:05:42.724287 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:05:42.724287 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:05:42.724287 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 09:05:42.724287 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 09:05:42.724287 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 09:05:42.724287 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 09:05:42.724287 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:05:42.724287 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:05:42.724287 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:05:42.724287 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:05:42.724287 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:05:42.724287 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:05:42.725166 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:05:42.725166 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 14 09:05:42.725166 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:05:42.725166 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:05:42.725166 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:05:42.725166 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:05:42.725166 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:05:42.725166 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:05:42.725166 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:05:42.725166 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:05:42.725166 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 09:05:42.725166 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 09:05:42.725166 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 14 09:05:42.725166 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 14 09:05:42.725166 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 494/1002] 10.222.0.26 () {56 vars in 1126 bytes} [Thu Jul 14 09:05:42 2022] GET /identity/v3/services/object-store => generated 93 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 09:05:42.730432 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e9d59a-d1a4-4623-a820-6237bebd699e None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:42.766860 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b741b3b-7d95-4bab-b775-069338a8e29e None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:42.802993 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afcd869-ca23-4f10-93f0-783afb9c5067 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:42.803795 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afcd869-ca23-4f10-93f0-783afb9c5067 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:42.804088 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afcd869-ca23-4f10-93f0-783afb9c5067 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:42.804357 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afcd869-ca23-4f10-93f0-783afb9c5067 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:42.805155 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afcd869-ca23-4f10-93f0-783afb9c5067 None admin] RBAC: Adding query filter params (interface=public, service_id=88658b970dd94b568e06e5dfbebf7b1b, region_id=RegionOne) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:05:42.805579 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afcd869-ca23-4f10-93f0-783afb9c5067 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:42.807062 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afcd869-ca23-4f10-93f0-783afb9c5067 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:42.817052 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 509/1003] 10.222.0.26 () {56 vars in 1251 bytes} [Thu Jul 14 09:05:42 2022] GET /identity/v3/endpoints?service_id=88658b970dd94b568e06e5dfbebf7b1b&interface=public®ion_id=RegionOne => generated 194 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:05:42.817918 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afcd869-ca23-4f10-93f0-783afb9c5067 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:42.844540 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e9d59a-d1a4-4623-a820-6237bebd699e None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:42.848381 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:42.873151 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:42.883214 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27074998-fc80-42f6-91a9-77baac0f707b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:42.883886 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27074998-fc80-42f6-91a9-77baac0f707b None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:42.884231 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27074998-fc80-42f6-91a9-77baac0f707b None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:42.884453 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27074998-fc80-42f6-91a9-77baac0f707b None admin] PATH_INFO: `/v3/services` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:42.885160 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27074998-fc80-42f6-91a9-77baac0f707b None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:05:42.885469 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27074998-fc80-42f6-91a9-77baac0f707b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:42.886843 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27074998-fc80-42f6-91a9-77baac0f707b None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:42.892245 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 495/1004] 10.222.0.26 () {56 vars in 1122 bytes} [Thu Jul 14 09:05:42 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 09:05:42.897656 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27074998-fc80-42f6-91a9-77baac0f707b None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:42.926019 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afcd869-ca23-4f10-93f0-783afb9c5067 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:42.965799 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd55db3-577f-4b57-b0e0-0bbd0c402435 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:42.966756 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd55db3-577f-4b57-b0e0-0bbd0c402435 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:42.967034 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd55db3-577f-4b57-b0e0-0bbd0c402435 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:42.967984 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd55db3-577f-4b57-b0e0-0bbd0c402435 None admin] PATH_INFO: `/v3/services` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:42.968677 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd55db3-577f-4b57-b0e0-0bbd0c402435 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:42.970551 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd55db3-577f-4b57-b0e0-0bbd0c402435 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:42.983660 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 510/1005] 10.222.0.26 () {56 vars in 1087 bytes} [Thu Jul 14 09:05:42 2022] GET /identity/v3/services => generated 1727 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 09:05:43.005698 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27074998-fc80-42f6-91a9-77baac0f707b None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:43.046014 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da12ab2e-6ef2-49d1-b148-4fda981986c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:43.046709 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da12ab2e-6ef2-49d1-b148-4fda981986c3 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:43.046963 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da12ab2e-6ef2-49d1-b148-4fda981986c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:43.047252 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da12ab2e-6ef2-49d1-b148-4fda981986c3 None admin] PATH_INFO: `/v3/services` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:43.048006 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da12ab2e-6ef2-49d1-b148-4fda981986c3 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:05:43.048293 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da12ab2e-6ef2-49d1-b148-4fda981986c3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:43.049718 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da12ab2e-6ef2-49d1-b148-4fda981986c3 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:43.056041 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 496/1006] 10.222.0.26 () {56 vars in 1122 bytes} [Thu Jul 14 09:05:42 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 09:05:43.061727 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd55db3-577f-4b57-b0e0-0bbd0c402435 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:43.168331 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:43.175840 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd55db3-577f-4b57-b0e0-0bbd0c402435 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:43.209537 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2aeaef6-5d0a-443a-a323-38012dc3c589 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:43.210202 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2aeaef6-5d0a-443a-a323-38012dc3c589 None admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:43.210494 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2aeaef6-5d0a-443a-a323-38012dc3c589 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:43.210772 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2aeaef6-5d0a-443a-a323-38012dc3c589 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:43.211564 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2aeaef6-5d0a-443a-a323-38012dc3c589 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:43.212942 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2aeaef6-5d0a-443a-a323-38012dc3c589 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:43.228867 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 511/1007] 10.222.0.26 () {60 vars in 1144 bytes} [Thu Jul 14 09:05:43 2022] POST /identity/v3/endpoints => generated 325 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 09:05:44.331801 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b2c1a6-91c1-4872-9193-9b93b82e432c None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:44.332329 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b2c1a6-91c1-4872-9193-9b93b82e432c None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:44.332329 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b2c1a6-91c1-4872-9193-9b93b82e432c None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:44.333756 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 497/1008] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:05:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:05:44.339101 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e72d7b1-26ec-4176-b70e-bf9a352f7b3d None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:44.339572 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e72d7b1-26ec-4176-b70e-bf9a352f7b3d None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:44.339660 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e72d7b1-26ec-4176-b70e-bf9a352f7b3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:44.356874 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-2e72d7b1-26ec-4176-b70e-bf9a352f7b3d None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:05:44.358114 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-2e72d7b1-26ec-4176-b70e-bf9a352f7b3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:05:44.439325 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 512/1009] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:05:44 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 09:05:44.457779 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b2c1a6-91c1-4872-9193-9b93b82e432c None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:44.517264 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:44.573279 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b2c1a6-91c1-4872-9193-9b93b82e432c None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:44.608554 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5842bfe-a2c0-4316-be4d-61d05ce44e7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:44.609136 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5842bfe-a2c0-4316-be4d-61d05ce44e7e None admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:44.609417 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5842bfe-a2c0-4316-be4d-61d05ce44e7e None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:44.609694 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5842bfe-a2c0-4316-be4d-61d05ce44e7e None admin] PATH_INFO: `/v3/services` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:44.610429 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5842bfe-a2c0-4316-be4d-61d05ce44e7e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:44.611835 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5842bfe-a2c0-4316-be4d-61d05ce44e7e None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:44.631229 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 498/1010] 10.222.0.26 () {60 vars in 1141 bytes} [Thu Jul 14 09:05:44 2022] POST /identity/v3/services => generated 242 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 09:05:44.991034 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c581809-9300-4226-9844-bd04593097db None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:44.991432 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c581809-9300-4226-9844-bd04593097db None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:44.991598 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c581809-9300-4226-9844-bd04593097db None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:44.993190 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 513/1011] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:05:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:05:44.998237 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1919380-497d-404d-8b23-b18505e58c87 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:44.999876 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1919380-497d-404d-8b23-b18505e58c87 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:44.999876 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1919380-497d-404d-8b23-b18505e58c87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:45.015678 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-b1919380-497d-404d-8b23-b18505e58c87 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:05:45.016862 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-b1919380-497d-404d-8b23-b18505e58c87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:05:45.091090 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 499/1012] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:05:44 2022] POST /identity/v3/auth/tokens => generated 1803 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 09:05:45.113210 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c581809-9300-4226-9844-bd04593097db None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:45.198037 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:45.221590 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c581809-9300-4226-9844-bd04593097db None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:45.228653 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae39ecf-6aca-4199-a37a-da7902691153 None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:45.228944 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae39ecf-6aca-4199-a37a-da7902691153 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:45.229227 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae39ecf-6aca-4199-a37a-da7902691153 None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:45.230748 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 500/1013] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:05:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:05:45.235532 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c218c2-8d95-428f-b3b7-22688da05d10 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:45.235823 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c218c2-8d95-428f-b3b7-22688da05d10 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:45.236104 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c218c2-8d95-428f-b3b7-22688da05d10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:45.252718 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-56c218c2-8d95-428f-b3b7-22688da05d10 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:05:45.253823 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-56c218c2-8d95-428f-b3b7-22688da05d10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:05:45.260642 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2381d26e-ff70-4397-be61-dc3a4c5a74bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:45.261858 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2381d26e-ff70-4397-be61-dc3a4c5a74bc None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:45.262144 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2381d26e-ff70-4397-be61-dc3a4c5a74bc None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:45.262423 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2381d26e-ff70-4397-be61-dc3a4c5a74bc None admin] PATH_INFO: `/v3/services/compute` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:45.263492 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2381d26e-ff70-4397-be61-dc3a4c5a74bc None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:45.265244 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2381d26e-ff70-4397-be61-dc3a4c5a74bc None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:45.278035 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-2381d26e-ff70-4397-be61-dc3a4c5a74bc None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 14 09:05:45.278035 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:05:45.278035 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 09:05:45.278035 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 09:05:45.278035 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 09:05:45.278035 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 09:05:45.278035 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 09:05:45.278035 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 09:05:45.278035 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 14 09:05:45.278035 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 14 09:05:45.278035 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 09:05:45.278035 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 14 09:05:45.278035 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:05:45.278035 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:05:45.278035 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:05:45.278035 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:05:45.278035 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:05:45.279444 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:05:45.279444 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:05:45.279444 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:05:45.279444 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:05:45.279444 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:05:45.279444 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:05:45.279444 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 09:05:45.279444 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 09:05:45.279444 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 09:05:45.279444 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 09:05:45.279444 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:05:45.279444 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:05:45.279444 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:05:45.279444 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:05:45.279444 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:05:45.279444 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:05:45.280384 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:05:45.280384 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 14 09:05:45.280384 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:05:45.280384 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:05:45.280384 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:05:45.280384 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:05:45.280384 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:05:45.280384 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:05:45.280384 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:05:45.280384 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:05:45.280384 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 09:05:45.280384 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 09:05:45.280384 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 14 09:05:45.280384 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 14 09:05:45.280384 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 514/1014] 10.222.0.26 () {56 vars in 1111 bytes} [Thu Jul 14 09:05:45 2022] GET /identity/v3/services/compute => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:05:45.285356 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2381d26e-ff70-4397-be61-dc3a4c5a74bc None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:45.294459 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:45.332723 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 501/1015] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:05:45 2022] POST /identity/v3/auth/tokens => generated 1803 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 09:05:45.355932 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c218c2-8d95-428f-b3b7-22688da05d10 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:45.393916 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2381d26e-ff70-4397-be61-dc3a4c5a74bc None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:45.428282 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c171dc81-94ea-4df1-b600-a62ab9cbc83d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:45.428946 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c171dc81-94ea-4df1-b600-a62ab9cbc83d None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:45.429232 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c171dc81-94ea-4df1-b600-a62ab9cbc83d None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:45.429511 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c171dc81-94ea-4df1-b600-a62ab9cbc83d None admin] PATH_INFO: `/v3/services` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:45.430212 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c171dc81-94ea-4df1-b600-a62ab9cbc83d None admin] RBAC: Adding query filter params (name=compute) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:05:45.430519 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c171dc81-94ea-4df1-b600-a62ab9cbc83d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:45.431930 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c171dc81-94ea-4df1-b600-a62ab9cbc83d None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:45.437055 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 515/1016] 10.222.0.26 () {56 vars in 1112 bytes} [Thu Jul 14 09:05:45 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 09:05:45.442840 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c171dc81-94ea-4df1-b600-a62ab9cbc83d None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:45.453874 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:45.463983 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c218c2-8d95-428f-b3b7-22688da05d10 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:45.500321 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5521a49-549f-4ab2-bd15-05158b06beac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:45.501121 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5521a49-549f-4ab2-bd15-05158b06beac None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:45.501404 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5521a49-549f-4ab2-bd15-05158b06beac None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:45.501680 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5521a49-549f-4ab2-bd15-05158b06beac None admin] PATH_INFO: `/v3/domains/default` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:45.502891 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5521a49-549f-4ab2-bd15-05158b06beac None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:45.504407 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5521a49-549f-4ab2-bd15-05158b06beac None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:45.511324 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 502/1017] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jul 14 09:05:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 09:05:45.512120 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5521a49-549f-4ab2-bd15-05158b06beac None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:45.553778 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c171dc81-94ea-4df1-b600-a62ab9cbc83d None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:45.558585 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:45.588068 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9e5488-233c-4af1-ab21-0e39e1821299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:45.588737 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9e5488-233c-4af1-ab21-0e39e1821299 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:45.589019 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9e5488-233c-4af1-ab21-0e39e1821299 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:45.589950 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9e5488-233c-4af1-ab21-0e39e1821299 None admin] PATH_INFO: `/v3/services` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:45.590072 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9e5488-233c-4af1-ab21-0e39e1821299 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:05:45.590291 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9e5488-233c-4af1-ab21-0e39e1821299 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:45.591693 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9e5488-233c-4af1-ab21-0e39e1821299 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:45.597273 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 516/1018] 10.222.0.26 () {56 vars in 1112 bytes} [Thu Jul 14 09:05:45 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 09:05:45.602941 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9e5488-233c-4af1-ab21-0e39e1821299 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:45.621038 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5521a49-549f-4ab2-bd15-05158b06beac None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:45.656846 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1735e10-6583-481b-971a-d8da14dfab2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:45.657935 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1735e10-6583-481b-971a-d8da14dfab2c None admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:45.658230 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1735e10-6583-481b-971a-d8da14dfab2c None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:45.658515 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1735e10-6583-481b-971a-d8da14dfab2c None admin] PATH_INFO: `/v3/projects` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:45.659408 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1735e10-6583-481b-971a-d8da14dfab2c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:45.660863 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1735e10-6583-481b-971a-d8da14dfab2c None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:45.676056 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:45.690357 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 503/1019] 10.222.0.26 () {60 vars in 1141 bytes} [Thu Jul 14 09:05:45 2022] POST /identity/v3/projects => generated 311 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 09:05:45.718736 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9e5488-233c-4af1-ab21-0e39e1821299 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:45.757365 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38955b22-6a74-441b-ad20-24e46e4a598b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:45.757992 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38955b22-6a74-441b-ad20-24e46e4a598b None admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:45.758265 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38955b22-6a74-441b-ad20-24e46e4a598b None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:45.758788 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38955b22-6a74-441b-ad20-24e46e4a598b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:45.759402 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38955b22-6a74-441b-ad20-24e46e4a598b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:45.760857 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38955b22-6a74-441b-ad20-24e46e4a598b None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:45.791221 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 517/1020] 10.222.0.26 () {60 vars in 1144 bytes} [Thu Jul 14 09:05:45 2022] POST /identity/v3/endpoints => generated 334 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 09:05:46.669337 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357d3cf4-837a-4f07-856f-789e4e487808 None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:46.669887 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357d3cf4-837a-4f07-856f-789e4e487808 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:46.669887 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357d3cf4-837a-4f07-856f-789e4e487808 None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:46.671593 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 504/1021] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:05:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:05:46.676617 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a914f9-0af2-4c38-8fb4-f09d22e4d714 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:46.677052 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a914f9-0af2-4c38-8fb4-f09d22e4d714 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:46.677129 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a914f9-0af2-4c38-8fb4-f09d22e4d714 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:46.694291 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-73a914f9-0af2-4c38-8fb4-f09d22e4d714 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:05:46.695489 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-73a914f9-0af2-4c38-8fb4-f09d22e4d714 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:05:46.769414 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 518/1022] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:05:46 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 09:05:46.792203 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357d3cf4-837a-4f07-856f-789e4e487808 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:46.885655 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:46.898836 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357d3cf4-837a-4f07-856f-789e4e487808 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:46.934323 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d407b8b5-23c1-4bf8-a1db-dfdafd06dfb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:46.935119 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d407b8b5-23c1-4bf8-a1db-dfdafd06dfb3 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:46.935521 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d407b8b5-23c1-4bf8-a1db-dfdafd06dfb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:46.935797 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d407b8b5-23c1-4bf8-a1db-dfdafd06dfb3 None admin] PATH_INFO: `/v3/services/image` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:46.936451 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d407b8b5-23c1-4bf8-a1db-dfdafd06dfb3 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:46.937821 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d407b8b5-23c1-4bf8-a1db-dfdafd06dfb3 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:46.949390 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-d407b8b5-23c1-4bf8-a1db-dfdafd06dfb3 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 14 09:05:46.949390 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:05:46.949390 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 09:05:46.949390 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 09:05:46.949390 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 09:05:46.949390 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 09:05:46.949390 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 09:05:46.949390 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 09:05:46.949390 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 14 09:05:46.949390 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 14 09:05:46.949390 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 09:05:46.949390 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 14 09:05:46.949390 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:05:46.949390 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:05:46.949390 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:05:46.949390 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:05:46.949390 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:05:46.950404 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:05:46.950404 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:05:46.950404 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:05:46.950404 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:05:46.950404 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:05:46.950404 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:05:46.950404 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 09:05:46.950404 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 09:05:46.950404 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 09:05:46.950404 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 09:05:46.950404 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:05:46.950404 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:05:46.950404 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:05:46.950404 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:05:46.950404 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:05:46.950404 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:05:46.951395 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:05:46.951395 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 14 09:05:46.951395 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:05:46.951395 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:05:46.951395 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:05:46.951395 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:05:46.951395 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:05:46.951395 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:05:46.951395 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:05:46.951395 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:05:46.951395 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 09:05:46.951395 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 09:05:46.951395 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 14 09:05:46.951395 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 14 09:05:46.951395 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 505/1023] 10.222.0.26 () {56 vars in 1105 bytes} [Thu Jul 14 09:05:46 2022] GET /identity/v3/services/image => generated 86 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:05:46.958604 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a914f9-0af2-4c38-8fb4-f09d22e4d714 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:47.015385 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:47.062635 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a914f9-0af2-4c38-8fb4-f09d22e4d714 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:47.097411 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4e9e65-046b-4775-a5dc-7698db9626e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:47.098063 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4e9e65-046b-4775-a5dc-7698db9626e8 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:47.098353 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4e9e65-046b-4775-a5dc-7698db9626e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:47.098616 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4e9e65-046b-4775-a5dc-7698db9626e8 None admin] PATH_INFO: `/v3/services` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:47.099364 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4e9e65-046b-4775-a5dc-7698db9626e8 None admin] RBAC: Adding query filter params (name=image) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:05:47.099671 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4e9e65-046b-4775-a5dc-7698db9626e8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:47.101044 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4e9e65-046b-4775-a5dc-7698db9626e8 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:47.107378 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 519/1024] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jul 14 09:05:46 2022] GET /identity/v3/services?name=image => generated 122 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:05:47.111797 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d407b8b5-23c1-4bf8-a1db-dfdafd06dfb3 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:47.218338 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d407b8b5-23c1-4bf8-a1db-dfdafd06dfb3 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:47.252731 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21bf142-bed9-4111-975b-f8402d5c71bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:47.253400 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21bf142-bed9-4111-975b-f8402d5c71bc None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:47.253686 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21bf142-bed9-4111-975b-f8402d5c71bc None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:47.253958 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21bf142-bed9-4111-975b-f8402d5c71bc None admin] PATH_INFO: `/v3/services` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:47.254673 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21bf142-bed9-4111-975b-f8402d5c71bc None admin] RBAC: Adding query filter params (type=image) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:05:47.254984 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21bf142-bed9-4111-975b-f8402d5c71bc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:47.256393 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21bf142-bed9-4111-975b-f8402d5c71bc None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:47.261667 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 506/1025] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jul 14 09:05:47 2022] GET /identity/v3/services?type=image => generated 350 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:05:47.267572 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4e9e65-046b-4775-a5dc-7698db9626e8 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:47.370059 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4e9e65-046b-4775-a5dc-7698db9626e8 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:47.387539 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:47.405844 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a42eda-04bd-4908-a295-498710758c48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:47.406506 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a42eda-04bd-4908-a295-498710758c48 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:47.406789 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a42eda-04bd-4908-a295-498710758c48 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:47.407061 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a42eda-04bd-4908-a295-498710758c48 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:47.407880 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a42eda-04bd-4908-a295-498710758c48 None admin] RBAC: Adding query filter params (interface=public, service_id=af88eba722084c9c9dd013b69e4e9e22, region_id=RegionOne) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:05:47.408186 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a42eda-04bd-4908-a295-498710758c48 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:47.409552 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a42eda-04bd-4908-a295-498710758c48 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:47.414879 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 520/1026] 10.222.0.26 () {56 vars in 1251 bytes} [Thu Jul 14 09:05:47 2022] GET /identity/v3/endpoints?service_id=af88eba722084c9c9dd013b69e4e9e22&interface=public®ion_id=RegionOne => generated 194 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:05:47.420386 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21bf142-bed9-4111-975b-f8402d5c71bc None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:47.441107 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:47.527613 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21bf142-bed9-4111-975b-f8402d5c71bc None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:47.568842 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6509f95d-a663-493b-aff8-64881a10aa1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:47.569515 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6509f95d-a663-493b-aff8-64881a10aa1f None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:47.569798 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6509f95d-a663-493b-aff8-64881a10aa1f None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:47.570074 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6509f95d-a663-493b-aff8-64881a10aa1f None admin] PATH_INFO: `/v3/services` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:47.570780 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6509f95d-a663-493b-aff8-64881a10aa1f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:47.572198 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6509f95d-a663-493b-aff8-64881a10aa1f None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:47.579335 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 507/1027] 10.222.0.26 () {56 vars in 1087 bytes} [Thu Jul 14 09:05:47 2022] GET /identity/v3/services => generated 1957 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 09:05:47.755159 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12033c46-89c2-425a-bd53-62a2a4060cfc None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:47.755783 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12033c46-89c2-425a-bd53-62a2a4060cfc None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:47.755783 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12033c46-89c2-425a-bd53-62a2a4060cfc None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:47.757101 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 521/1028] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:05:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:05:47.762528 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3dd1c5-1b07-4dca-b3ce-651231e4e56f None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:47.762975 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3dd1c5-1b07-4dca-b3ce-651231e4e56f None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:47.763073 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3dd1c5-1b07-4dca-b3ce-651231e4e56f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:47.780611 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-2c3dd1c5-1b07-4dca-b3ce-651231e4e56f None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:05:47.781757 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-2c3dd1c5-1b07-4dca-b3ce-651231e4e56f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:05:47.851385 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 508/1029] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:05:47 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 09:05:47.874987 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12033c46-89c2-425a-bd53-62a2a4060cfc None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:47.889709 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e91e45-3402-4662-9310-81a274615506 None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:47.890014 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e91e45-3402-4662-9310-81a274615506 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:47.890272 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e91e45-3402-4662-9310-81a274615506 None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:47.891837 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 509/1030] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:05:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 09:05:47.896615 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed1b104-630e-4b62-9762-abb9b20a674d None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:47.896905 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed1b104-630e-4b62-9762-abb9b20a674d None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:47.897185 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed1b104-630e-4b62-9762-abb9b20a674d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:47.914734 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-9ed1b104-630e-4b62-9762-abb9b20a674d None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:05:47.915967 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-9ed1b104-630e-4b62-9762-abb9b20a674d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:05:47.924245 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:47.988251 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 510/1031] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:05:47 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 09:05:47.991181 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12033c46-89c2-425a-bd53-62a2a4060cfc None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:48.012173 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed1b104-630e-4b62-9762-abb9b20a674d None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:48.026899 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e8493e-0833-4877-9219-322a0ca98fa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:48.028115 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e8493e-0833-4877-9219-322a0ca98fa7 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:48.028693 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e8493e-0833-4877-9219-322a0ca98fa7 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:48.029364 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e8493e-0833-4877-9219-322a0ca98fa7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:48.030592 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e8493e-0833-4877-9219-322a0ca98fa7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:48.032126 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e8493e-0833-4877-9219-322a0ca98fa7 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:48.034726 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 522/1032] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jul 14 09:05:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:05:48.039423 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e8493e-0833-4877-9219-322a0ca98fa7 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:48.094301 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:48.118567 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed1b104-630e-4b62-9762-abb9b20a674d None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:48.146474 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e8493e-0833-4877-9219-322a0ca98fa7 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:48.155403 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbd4402-8643-4aa1-bb88-1a33a76a3812 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:48.156229 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbd4402-8643-4aa1-bb88-1a33a76a3812 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:48.156669 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbd4402-8643-4aa1-bb88-1a33a76a3812 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:48.156881 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbd4402-8643-4aa1-bb88-1a33a76a3812 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:48.167377 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbd4402-8643-4aa1-bb88-1a33a76a3812 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 09:05:48.175585 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-1cbd4402-8643-4aa1-bb88-1a33a76a3812 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 14 09:05:48.175585 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:05:48.175585 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:05:48.175585 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:05:48.175585 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:05:48.175585 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:05:48.175585 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:05:48.175585 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:05:48.175585 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:05:48.175585 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:05:48.175585 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:05:48.175585 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:05:48.175585 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 09:05:48.175585 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 09:05:48.175585 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 09:05:48.175585 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 09:05:48.176542 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 09:05:48.176542 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 09:05:48.176542 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 09:05:48.176542 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 09:05:48.176542 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:05:48.176542 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:05:48.176542 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:05:48.176542 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:05:48.176542 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:05:48.176542 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:05:48.176542 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:05:48.176542 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 14 09:05:48.176542 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:05:48.176542 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:05:48.176542 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:05:48.176542 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:05:48.177410 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:05:48.177410 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:05:48.177410 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:05:48.177410 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:05:48.177410 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 09:05:48.177410 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 09:05:48.177410 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 09:05:48.177410 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 09:05:48.177410 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 09:05:48.177410 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 09:05:48.177410 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 14 09:05:48.177410 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 14 09:05:48.177410 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 511/1033] 10.222.0.26 () {56 vars in 1120 bytes} [Thu Jul 14 09:05:48 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 09:05:48.182581 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbd4402-8643-4aa1-bb88-1a33a76a3812 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:48.184199 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9634f0b0-0aff-4601-b089-483b8a6496eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:48.184846 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9634f0b0-0aff-4601-b089-483b8a6496eb None admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:48.185300 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9634f0b0-0aff-4601-b089-483b8a6496eb None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:48.185456 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9634f0b0-0aff-4601-b089-483b8a6496eb None admin] PATH_INFO: `/v3/users` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:48.186162 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9634f0b0-0aff-4601-b089-483b8a6496eb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:48.187406 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:48.187729 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9634f0b0-0aff-4601-b089-483b8a6496eb None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:48.209927 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 523/1034] 10.222.0.26 () {60 vars in 1132 bytes} [Thu Jul 14 09:05:48 2022] POST /identity/v3/users => generated 285 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 09:05:48.300011 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbd4402-8643-4aa1-bb88-1a33a76a3812 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:48.336632 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e72fc2-405a-4b4a-b439-83e2966cb958 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:48.337314 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e72fc2-405a-4b4a-b439-83e2966cb958 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:48.337597 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e72fc2-405a-4b4a-b439-83e2966cb958 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:48.337877 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e72fc2-405a-4b4a-b439-83e2966cb958 None admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:48.338642 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e72fc2-405a-4b4a-b439-83e2966cb958 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:05:48.338952 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e72fc2-405a-4b4a-b439-83e2966cb958 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:48.340393 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e72fc2-405a-4b4a-b439-83e2966cb958 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:48.349939 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 512/1035] 10.222.0.26 () {56 vars in 1115 bytes} [Thu Jul 14 09:05:48 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 09:05:48.355536 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9634f0b0-0aff-4601-b089-483b8a6496eb None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:48.468869 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9634f0b0-0aff-4601-b089-483b8a6496eb None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:48.476806 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:48.507900 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de87916-9b11-4636-9554-6ad4ef7e0a34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:48.508736 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de87916-9b11-4636-9554-6ad4ef7e0a34 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:48.509023 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de87916-9b11-4636-9554-6ad4ef7e0a34 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:48.509299 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de87916-9b11-4636-9554-6ad4ef7e0a34 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:48.520194 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de87916-9b11-4636-9554-6ad4ef7e0a34 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:48.521649 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de87916-9b11-4636-9554-6ad4ef7e0a34 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:48.542538 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-3de87916-9b11-4636-9554-6ad4ef7e0a34 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 09:05:48.542538 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:05:48.542538 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:05:48.542538 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:05:48.542538 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:05:48.542538 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:05:48.542538 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:05:48.542538 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:05:48.542538 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:05:48.542538 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:05:48.542538 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:05:48.542538 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:05:48.542538 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 09:05:48.542538 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 09:05:48.542538 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 09:05:48.543510 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 09:05:48.543510 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:05:48.543510 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:05:48.543510 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:05:48.543510 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:05:48.543510 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:05:48.543510 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:05:48.543510 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:05:48.543510 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 14 09:05:48.543510 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:05:48.543510 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:05:48.543510 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:05:48.543510 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:05:48.543510 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:05:48.543510 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:05:48.543510 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:05:48.543510 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:05:48.544524 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 09:05:48.544524 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 09:05:48.544524 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 09:05:48.544524 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 14 09:05:48.544524 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 524/1036] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jul 14 09:05:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:05:48.552831 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e72fc2-405a-4b4a-b439-83e2966cb958 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:48.617365 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:48.659434 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e72fc2-405a-4b4a-b439-83e2966cb958 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:48.698543 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a113a43-9f2a-404e-8e9d-caea6eb13d37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:48.699218 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a113a43-9f2a-404e-8e9d-caea6eb13d37 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:48.699505 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a113a43-9f2a-404e-8e9d-caea6eb13d37 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:48.699771 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a113a43-9f2a-404e-8e9d-caea6eb13d37 None admin] PATH_INFO: `/v3/domains` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:48.700592 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a113a43-9f2a-404e-8e9d-caea6eb13d37 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:05:48.700859 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a113a43-9f2a-404e-8e9d-caea6eb13d37 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:48.702221 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a113a43-9f2a-404e-8e9d-caea6eb13d37 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:48.719301 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 513/1037] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Jul 14 09:05:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:05:48.721007 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de87916-9b11-4636-9554-6ad4ef7e0a34 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:48.828594 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de87916-9b11-4636-9554-6ad4ef7e0a34 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:48.864332 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2eeaec-d929-40e8-8cda-3ff3bdb17c07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:48.865176 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2eeaec-d929-40e8-8cda-3ff3bdb17c07 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:48.865493 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2eeaec-d929-40e8-8cda-3ff3bdb17c07 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:48.865770 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2eeaec-d929-40e8-8cda-3ff3bdb17c07 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:48.872957 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2eeaec-d929-40e8-8cda-3ff3bdb17c07 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:48.874391 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2eeaec-d929-40e8-8cda-3ff3bdb17c07 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:48.888815 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-5e2eeaec-d929-40e8-8cda-3ff3bdb17c07 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 14 09:05:48.888815 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:05:48.888815 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:05:48.888815 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:05:48.888815 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:05:48.888815 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:05:48.888815 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:05:48.888815 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:05:48.888815 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:05:48.888815 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:05:48.888815 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:05:48.888815 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:05:48.888815 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 09:05:48.888815 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 09:05:48.888815 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 09:05:48.888815 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 09:05:48.889778 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:05:48.889778 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:05:48.889778 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 09:05:48.889778 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 09:05:48.889778 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 09:05:48.889778 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 09:05:48.889778 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:05:48.889778 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:05:48.889778 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:05:48.889778 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:05:48.889778 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:05:48.889778 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 14 09:05:48.889778 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:05:48.889778 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:05:48.889778 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:05:48.889778 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:05:48.889778 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:05:48.889778 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:05:48.890749 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:05:48.890749 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:05:48.890749 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 09:05:48.890749 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 09:05:48.890749 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 09:05:48.890749 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 09:05:48.890749 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 09:05:48.890749 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 09:05:48.890749 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 14 09:05:48.890749 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 14 09:05:48.894767 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 525/1038] 10.222.0.26 () {56 vars in 1093 bytes} [Thu Jul 14 09:05:48 2022] GET /identity/v3/users/nova => generated 82 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:05:48.895850 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a113a43-9f2a-404e-8e9d-caea6eb13d37 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:48.993537 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a113a43-9f2a-404e-8e9d-caea6eb13d37 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:49.017064 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:49.028924 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4c70d0-106a-49d1-a83c-7561de7d4950 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:49.029588 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4c70d0-106a-49d1-a83c-7561de7d4950 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:49.029871 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4c70d0-106a-49d1-a83c-7561de7d4950 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:49.030146 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4c70d0-106a-49d1-a83c-7561de7d4950 None admin] PATH_INFO: `/v3/users` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:49.030991 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4c70d0-106a-49d1-a83c-7561de7d4950 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:05:49.031325 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4c70d0-106a-49d1-a83c-7561de7d4950 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:49.032766 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4c70d0-106a-49d1-a83c-7561de7d4950 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:49.053697 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 514/1039] 10.222.0.26 () {56 vars in 1133 bytes} [Thu Jul 14 09:05:48 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:05:49.059879 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2eeaec-d929-40e8-8cda-3ff3bdb17c07 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:49.080739 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:49.178412 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2eeaec-d929-40e8-8cda-3ff3bdb17c07 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:49.215399 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6ce6de-710b-432d-ba7e-c33c63c3e459 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:49.216111 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6ce6de-710b-432d-ba7e-c33c63c3e459 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:49.216373 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6ce6de-710b-432d-ba7e-c33c63c3e459 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:49.216666 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6ce6de-710b-432d-ba7e-c33c63c3e459 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:49.285402 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6ce6de-710b-432d-ba7e-c33c63c3e459 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:49.286918 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6ce6de-710b-432d-ba7e-c33c63c3e459 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:49.305848 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:49.386432 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 526/1040] 10.222.0.26 () {58 vars in 1282 bytes} [Thu Jul 14 09:05:49 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 09:05:49.392027 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4c70d0-106a-49d1-a83c-7561de7d4950 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:49.493680 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4c70d0-106a-49d1-a83c-7561de7d4950 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:49.527233 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5473331c-7153-4095-963c-b5cd68550c48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:49.528331 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5473331c-7153-4095-963c-b5cd68550c48 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:49.528638 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5473331c-7153-4095-963c-b5cd68550c48 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:49.528909 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5473331c-7153-4095-963c-b5cd68550c48 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:49.544623 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5473331c-7153-4095-963c-b5cd68550c48 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:49.546071 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5473331c-7153-4095-963c-b5cd68550c48 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:49.564950 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-5473331c-7153-4095-963c-b5cd68550c48 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 09:05:49.564950 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:05:49.564950 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:05:49.564950 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:05:49.564950 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:05:49.564950 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:05:49.564950 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:05:49.564950 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:05:49.564950 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:05:49.564950 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:05:49.564950 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:05:49.564950 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:05:49.564950 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 09:05:49.564950 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 09:05:49.564950 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 09:05:49.565903 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 09:05:49.565903 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:05:49.565903 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:05:49.565903 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:05:49.565903 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:05:49.565903 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:05:49.565903 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:05:49.565903 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:05:49.565903 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 14 09:05:49.565903 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:05:49.565903 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:05:49.565903 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:05:49.565903 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:05:49.565903 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:05:49.565903 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:05:49.565903 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:05:49.565903 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:05:49.566821 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 09:05:49.566821 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 09:05:49.566821 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 09:05:49.566821 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 14 09:05:49.567733 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 515/1041] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jul 14 09:05:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:05:49.571860 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6ce6de-710b-432d-ba7e-c33c63c3e459 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:49.601783 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e3409c-f6f5-44e4-9dd8-a4a87ca6f9c7 None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:49.602091 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e3409c-f6f5-44e4-9dd8-a4a87ca6f9c7 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:49.602324 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e3409c-f6f5-44e4-9dd8-a4a87ca6f9c7 None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:49.604078 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 516/1042] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:05:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 09:05:49.608787 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b910eb89-d4d9-4746-a827-4e1f6b7dc609 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:49.609074 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b910eb89-d4d9-4746-a827-4e1f6b7dc609 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:49.609351 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b910eb89-d4d9-4746-a827-4e1f6b7dc609 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:49.630194 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-b910eb89-d4d9-4746-a827-4e1f6b7dc609 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:05:49.631651 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-b910eb89-d4d9-4746-a827-4e1f6b7dc609 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:05:49.661501 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:49.675659 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6ce6de-710b-432d-ba7e-c33c63c3e459 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:49.695883 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:49.707599 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 517/1043] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:05:49 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 09:05:49.719818 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bf03f5-2c87-4de3-9bd7-bb859c1350c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:49.720489 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bf03f5-2c87-4de3-9bd7-bb859c1350c5 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:49.720771 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bf03f5-2c87-4de3-9bd7-bb859c1350c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:49.721045 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bf03f5-2c87-4de3-9bd7-bb859c1350c5 None admin] PATH_INFO: `/v3/domains` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:49.721769 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bf03f5-2c87-4de3-9bd7-bb859c1350c5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:05:49.722166 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bf03f5-2c87-4de3-9bd7-bb859c1350c5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:49.723594 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bf03f5-2c87-4de3-9bd7-bb859c1350c5 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:49.731134 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b910eb89-d4d9-4746-a827-4e1f6b7dc609 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:49.739536 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 527/1044] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Jul 14 09:05:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 09:05:49.741401 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bf03f5-2c87-4de3-9bd7-bb859c1350c5 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:49.834082 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b910eb89-d4d9-4746-a827-4e1f6b7dc609 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:49.854347 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bf03f5-2c87-4de3-9bd7-bb859c1350c5 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:49.871564 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7c1da3-6987-498e-b309-52998ee89787 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:49.872420 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7c1da3-6987-498e-b309-52998ee89787 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:49.872690 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7c1da3-6987-498e-b309-52998ee89787 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:49.872966 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7c1da3-6987-498e-b309-52998ee89787 None admin] PATH_INFO: `/v3/services/image` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:49.873620 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7c1da3-6987-498e-b309-52998ee89787 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:49.874994 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7c1da3-6987-498e-b309-52998ee89787 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:49.887988 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-8d7c1da3-6987-498e-b309-52998ee89787 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 14 09:05:49.887988 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:05:49.887988 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 09:05:49.887988 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 09:05:49.887988 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 09:05:49.887988 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 09:05:49.887988 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 09:05:49.887988 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 09:05:49.887988 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 14 09:05:49.887988 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 14 09:05:49.887988 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 09:05:49.887988 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 14 09:05:49.887988 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:05:49.887988 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:05:49.887988 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:05:49.887988 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:05:49.887988 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:05:49.889032 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:05:49.889032 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:05:49.889032 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:05:49.889032 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:05:49.889032 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:05:49.889032 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:05:49.889032 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 09:05:49.889032 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 09:05:49.889032 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 09:05:49.889032 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 09:05:49.889032 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:05:49.889032 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:05:49.889032 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:05:49.889032 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:05:49.889032 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:05:49.889032 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:05:49.889888 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:05:49.889888 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 14 09:05:49.889888 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:05:49.889888 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:05:49.889888 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:05:49.889888 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:05:49.889888 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:05:49.889888 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:05:49.889888 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:05:49.889888 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:05:49.889888 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 09:05:49.889888 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 09:05:49.889888 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 14 09:05:49.889888 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 14 09:05:49.891233 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d98122-18e2-48e2-82eb-6a16695a29ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:49.891233 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d98122-18e2-48e2-82eb-6a16695a29ba None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:49.891233 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d98122-18e2-48e2-82eb-6a16695a29ba None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:49.891233 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d98122-18e2-48e2-82eb-6a16695a29ba None admin] PATH_INFO: `/v3/projects/service` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:49.894592 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 518/1045] 10.222.0.26 () {56 vars in 1105 bytes} [Thu Jul 14 09:05:49 2022] GET /identity/v3/services/image => generated 86 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 09:05:49.895303 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7c1da3-6987-498e-b309-52998ee89787 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:49.897569 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d98122-18e2-48e2-82eb-6a16695a29ba None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:49.899007 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d98122-18e2-48e2-82eb-6a16695a29ba None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:49.913541 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-d9d98122-18e2-48e2-82eb-6a16695a29ba None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 09:05:49.913541 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:05:49.913541 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:05:49.913541 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:05:49.913541 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:05:49.913541 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:05:49.913541 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:05:49.913541 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:05:49.913541 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:05:49.913541 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:05:49.913541 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:05:49.913541 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:05:49.913541 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 09:05:49.913541 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 09:05:49.913541 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 09:05:49.914469 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 09:05:49.914469 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:05:49.914469 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:05:49.914469 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:05:49.914469 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:05:49.914469 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:05:49.914469 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:05:49.914469 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:05:49.914469 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 14 09:05:49.914469 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:05:49.914469 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:05:49.914469 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:05:49.914469 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:05:49.914469 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:05:49.914469 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:05:49.914469 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:05:49.914469 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:05:49.915395 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 09:05:49.915395 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 09:05:49.915395 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 09:05:49.915395 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 09:05:49.915395 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 09:05:49.915395 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 09:05:49.915395 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 09:05:49.915395 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 14 09:05:49.915395 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 528/1046] 10.222.0.26 () {56 vars in 1111 bytes} [Thu Jul 14 09:05:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 09:05:49.920361 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d98122-18e2-48e2-82eb-6a16695a29ba None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:49.928416 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:49.963795 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:50.007572 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7c1da3-6987-498e-b309-52998ee89787 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:50.027105 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d98122-18e2-48e2-82eb-6a16695a29ba None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:50.043658 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2b5a19-f20b-4005-9f5a-6e69ba63de0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:50.044418 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2b5a19-f20b-4005-9f5a-6e69ba63de0f None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:50.044720 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2b5a19-f20b-4005-9f5a-6e69ba63de0f None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:50.044968 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2b5a19-f20b-4005-9f5a-6e69ba63de0f None admin] PATH_INFO: `/v3/services` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:50.045668 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2b5a19-f20b-4005-9f5a-6e69ba63de0f None admin] RBAC: Adding query filter params (name=image) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:05:50.045973 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2b5a19-f20b-4005-9f5a-6e69ba63de0f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:50.047357 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2b5a19-f20b-4005-9f5a-6e69ba63de0f None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:50.052562 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 519/1047] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jul 14 09:05:49 2022] GET /identity/v3/services?name=image => generated 122 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:05:50.058055 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2b5a19-f20b-4005-9f5a-6e69ba63de0f None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:50.062980 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c79825-e144-4931-864c-6fdab6e6a952 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:50.063667 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c79825-e144-4931-864c-6fdab6e6a952 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:50.063948 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c79825-e144-4931-864c-6fdab6e6a952 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:50.064220 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c79825-e144-4931-864c-6fdab6e6a952 None admin] PATH_INFO: `/v3/projects` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:50.065009 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c79825-e144-4931-864c-6fdab6e6a952 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:05:50.065314 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c79825-e144-4931-864c-6fdab6e6a952 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:50.066739 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c79825-e144-4931-864c-6fdab6e6a952 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:50.083592 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 529/1048] 10.222.0.26 () {56 vars in 1148 bytes} [Thu Jul 14 09:05:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 09:05:50.084692 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c79825-e144-4931-864c-6fdab6e6a952 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:50.154606 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:50.164723 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2b5a19-f20b-4005-9f5a-6e69ba63de0f None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:50.184853 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:50.188658 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c79825-e144-4931-864c-6fdab6e6a952 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:50.200252 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da42a7e2-af7a-40f4-a462-605e91f9b213 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:50.200924 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da42a7e2-af7a-40f4-a462-605e91f9b213 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:50.201206 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da42a7e2-af7a-40f4-a462-605e91f9b213 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:50.201474 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da42a7e2-af7a-40f4-a462-605e91f9b213 None admin] PATH_INFO: `/v3/services` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:50.202179 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da42a7e2-af7a-40f4-a462-605e91f9b213 None admin] RBAC: Adding query filter params (type=image) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:05:50.202484 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da42a7e2-af7a-40f4-a462-605e91f9b213 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:50.204183 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da42a7e2-af7a-40f4-a462-605e91f9b213 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:50.213740 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 520/1049] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jul 14 09:05:50 2022] GET /identity/v3/services?type=image => generated 350 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 09:05:50.214819 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da42a7e2-af7a-40f4-a462-605e91f9b213 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:50.224369 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb40ef9-076d-42db-a06d-f6a9bd41c373 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:50.225034 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb40ef9-076d-42db-a06d-f6a9bd41c373 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:50.225319 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb40ef9-076d-42db-a06d-f6a9bd41c373 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:50.226097 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb40ef9-076d-42db-a06d-f6a9bd41c373 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:50.226942 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb40ef9-076d-42db-a06d-f6a9bd41c373 None admin] RBAC: Adding query filter params (role.id=70488f05098d4ce09e2fb03aaaea4566, scope.project.id=33bf210d015847439655b9943d1b0261, user.id=538fff3c5a2b487eb44ae7c1a1198102) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:05:50.227286 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb40ef9-076d-42db-a06d-f6a9bd41c373 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:50.228725 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb40ef9-076d-42db-a06d-f6a9bd41c373 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:50.234403 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 530/1050] 10.222.0.26 () {56 vars in 1374 bytes} [Thu Jul 14 09:05:50 2022] GET /identity/v3/role_assignments?user.id=538fff3c5a2b487eb44ae7c1a1198102&scope.project.id=33bf210d015847439655b9943d1b0261&role.id=70488f05098d4ce09e2fb03aaaea4566 => generated 259 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:05:50.314047 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18653f8c-3d01-43b6-ad11-0fcd07f04145 None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:50.314356 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18653f8c-3d01-43b6-ad11-0fcd07f04145 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:50.314642 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18653f8c-3d01-43b6-ad11-0fcd07f04145 None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:50.316237 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 531/1051] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:05:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 09:05:50.321741 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1195558a-3071-4995-80c9-cdc18ebe85c1 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:50.322018 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1195558a-3071-4995-80c9-cdc18ebe85c1 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:50.322314 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1195558a-3071-4995-80c9-cdc18ebe85c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:50.324125 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da42a7e2-af7a-40f4-a462-605e91f9b213 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:50.339651 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-1195558a-3071-4995-80c9-cdc18ebe85c1 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:05:50.341214 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-1195558a-3071-4995-80c9-cdc18ebe85c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:05:50.359881 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a00100-71e0-429b-ab4c-487efe0d40b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:50.360551 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a00100-71e0-429b-ab4c-487efe0d40b6 None admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:50.360833 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a00100-71e0-429b-ab4c-487efe0d40b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:50.361108 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a00100-71e0-429b-ab4c-487efe0d40b6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:50.361866 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a00100-71e0-429b-ab4c-487efe0d40b6 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:50.363286 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a00100-71e0-429b-ab4c-487efe0d40b6 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:50.385239 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 521/1052] 10.222.0.26 () {60 vars in 1144 bytes} [Thu Jul 14 09:05:50 2022] POST /identity/v3/endpoints => generated 327 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 09:05:50.435431 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 532/1053] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:05:50 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 09:05:50.458757 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a00100-71e0-429b-ab4c-487efe0d40b6 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:50.478506 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:50.582885 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a00100-71e0-429b-ab4c-487efe0d40b6 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:50.618545 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0cbc91-8ff3-47f9-9162-37bace436cb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:50.619368 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0cbc91-8ff3-47f9-9162-37bace436cb1 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:50.619648 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0cbc91-8ff3-47f9-9162-37bace436cb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:50.619920 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0cbc91-8ff3-47f9-9162-37bace436cb1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:50.630388 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0cbc91-8ff3-47f9-9162-37bace436cb1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 09:05:50.638006 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-fa0cbc91-8ff3-47f9-9162-37bace436cb1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 09:05:50.638006 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:05:50.638006 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:05:50.638006 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:05:50.638006 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:05:50.638006 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:05:50.638006 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:05:50.638006 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:05:50.638006 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:05:50.638006 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:05:50.638006 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:05:50.638006 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:05:50.638006 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 09:05:50.638006 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 09:05:50.638006 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 09:05:50.638006 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 09:05:50.638992 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 09:05:50.638992 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 09:05:50.638992 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 09:05:50.638992 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 09:05:50.638992 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:05:50.638992 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:05:50.638992 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:05:50.638992 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:05:50.638992 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:05:50.638992 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:05:50.638992 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:05:50.638992 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 14 09:05:50.638992 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:05:50.638992 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:05:50.638992 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:05:50.638992 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:05:50.644549 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:05:50.644549 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:05:50.644549 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:05:50.644549 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:05:50.644549 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 09:05:50.644549 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 09:05:50.644549 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 09:05:50.644549 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 09:05:50.644549 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 09:05:50.644549 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 09:05:50.644549 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 09:05:50.644549 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 14 09:05:50.644549 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 522/1054] 10.222.0.26 () {56 vars in 1096 bytes} [Thu Jul 14 09:05:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:05:50.648327 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1195558a-3071-4995-80c9-cdc18ebe85c1 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:50.742728 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:50.756126 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1195558a-3071-4995-80c9-cdc18ebe85c1 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:50.797223 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1102ff-12dc-4e8f-bb55-0835be28caca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:50.797870 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1102ff-12dc-4e8f-bb55-0835be28caca None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:50.798184 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1102ff-12dc-4e8f-bb55-0835be28caca None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:50.798435 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1102ff-12dc-4e8f-bb55-0835be28caca None admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:50.799230 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1102ff-12dc-4e8f-bb55-0835be28caca None admin] RBAC: Adding query filter params (name=admin) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:05:50.799549 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1102ff-12dc-4e8f-bb55-0835be28caca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:50.800924 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1102ff-12dc-4e8f-bb55-0835be28caca None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:50.809659 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 533/1055] 10.222.0.26 () {56 vars in 1099 bytes} [Thu Jul 14 09:05:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:05:50.815438 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0cbc91-8ff3-47f9-9162-37bace436cb1 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:50.893269 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:50.925064 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0cbc91-8ff3-47f9-9162-37bace436cb1 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:50.960645 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399279b5-fba7-4932-9f45-c01825943e85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:50.961517 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399279b5-fba7-4932-9f45-c01825943e85 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:50.961796 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399279b5-fba7-4932-9f45-c01825943e85 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:50.962068 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399279b5-fba7-4932-9f45-c01825943e85 None admin] PATH_INFO: `/v3/users/02d4fb8036934bd9a5596addd9739d43` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:50.976088 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399279b5-fba7-4932-9f45-c01825943e85 None admin] RBAC: Authorizing `identity:get_user(user_id=02d4fb8036934bd9a5596addd9739d43)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:50.977538 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399279b5-fba7-4932-9f45-c01825943e85 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:50.979419 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 523/1056] 10.222.0.26 () {56 vars in 1177 bytes} [Thu Jul 14 09:05:50 2022] GET /identity/v3/users/02d4fb8036934bd9a5596addd9739d43 => generated 285 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:05:50.985048 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1102ff-12dc-4e8f-bb55-0835be28caca None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:51.085267 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1102ff-12dc-4e8f-bb55-0835be28caca None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:51.119493 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1938f09d-d597-452e-9a1b-b53cfdf9fb33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:51.120144 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1938f09d-d597-452e-9a1b-b53cfdf9fb33 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:51.120424 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1938f09d-d597-452e-9a1b-b53cfdf9fb33 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:51.120670 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1938f09d-d597-452e-9a1b-b53cfdf9fb33 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:51.139492 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:51.195313 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1938f09d-d597-452e-9a1b-b53cfdf9fb33 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:51.196749 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1938f09d-d597-452e-9a1b-b53cfdf9fb33 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:51.293964 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 534/1057] 10.222.0.26 () {58 vars in 1282 bytes} [Thu Jul 14 09:05:50 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 09:05:51.300101 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399279b5-fba7-4932-9f45-c01825943e85 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:51.400729 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399279b5-fba7-4932-9f45-c01825943e85 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:51.428901 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:51.435923 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d853cd1-11fb-4ca4-b00e-275458d06e05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:51.436781 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d853cd1-11fb-4ca4-b00e-275458d06e05 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:51.437060 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d853cd1-11fb-4ca4-b00e-275458d06e05 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:51.437331 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d853cd1-11fb-4ca4-b00e-275458d06e05 None admin] PATH_INFO: `/v3/projects/d1b7367661964cf5a35f29aebdfa4650` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:51.438502 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d853cd1-11fb-4ca4-b00e-275458d06e05 None admin] RBAC: Authorizing `identity:get_project(project_id=d1b7367661964cf5a35f29aebdfa4650)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:51.439957 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d853cd1-11fb-4ca4-b00e-275458d06e05 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:51.441815 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 524/1058] 10.222.0.26 () {56 vars in 1186 bytes} [Thu Jul 14 09:05:51 2022] GET /identity/v3/projects/d1b7367661964cf5a35f29aebdfa4650 => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:05:51.447732 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1938f09d-d597-452e-9a1b-b53cfdf9fb33 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:51.497722 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:51.556811 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1938f09d-d597-452e-9a1b-b53cfdf9fb33 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:51.590656 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b3b4fb-91ad-4fcb-be60-be26092ca23f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:51.591344 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b3b4fb-91ad-4fcb-be60-be26092ca23f None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:51.591620 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b3b4fb-91ad-4fcb-be60-be26092ca23f None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:51.591887 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b3b4fb-91ad-4fcb-be60-be26092ca23f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:51.592710 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b3b4fb-91ad-4fcb-be60-be26092ca23f None admin] RBAC: Adding query filter params (role.id=7b549d58d1724395b343609d895c80d0, scope.project.id=d1b7367661964cf5a35f29aebdfa4650, user.id=02d4fb8036934bd9a5596addd9739d43) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:05:51.593018 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b3b4fb-91ad-4fcb-be60-be26092ca23f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:51.594434 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b3b4fb-91ad-4fcb-be60-be26092ca23f None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:51.600063 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 535/1059] 10.222.0.26 () {56 vars in 1374 bytes} [Thu Jul 14 09:05:51 2022] GET /identity/v3/role_assignments?user.id=02d4fb8036934bd9a5596addd9739d43&scope.project.id=d1b7367661964cf5a35f29aebdfa4650&role.id=7b549d58d1724395b343609d895c80d0 => generated 259 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:05:52.216122 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787e3999-7b6e-4bf2-bf9b-a7be598d4ec2 None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:52.219026 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787e3999-7b6e-4bf2-bf9b-a7be598d4ec2 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:52.219026 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787e3999-7b6e-4bf2-bf9b-a7be598d4ec2 None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:52.219026 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 525/1060] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:05:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:05:52.223415 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4f1144-34f6-404e-a050-4144a016a9c3 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:52.223934 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4f1144-34f6-404e-a050-4144a016a9c3 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:52.223934 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4f1144-34f6-404e-a050-4144a016a9c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:52.241121 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-2c4f1144-34f6-404e-a050-4144a016a9c3 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:05:52.242285 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-2c4f1144-34f6-404e-a050-4144a016a9c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:05:52.312037 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 536/1061] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:05:52 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 09:05:52.333961 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787e3999-7b6e-4bf2-bf9b-a7be598d4ec2 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:52.417497 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0733602c-9094-4a9b-b1a7-ca04b21a6604 None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:52.417817 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0733602c-9094-4a9b-b1a7-ca04b21a6604 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:52.418044 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0733602c-9094-4a9b-b1a7-ca04b21a6604 None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:52.419542 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 537/1062] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:05:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:05:52.424521 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908efebf-cc01-464f-8583-64e8b7d4cc24 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:52.424787 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908efebf-cc01-464f-8583-64e8b7d4cc24 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:52.425083 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908efebf-cc01-464f-8583-64e8b7d4cc24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:52.439769 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787e3999-7b6e-4bf2-bf9b-a7be598d4ec2 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:52.442001 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-908efebf-cc01-464f-8583-64e8b7d4cc24 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:05:52.443175 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-908efebf-cc01-464f-8583-64e8b7d4cc24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:05:52.474764 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03dbd7c-29b5-430d-afdc-a0d5b9cde659 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:52.475890 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03dbd7c-29b5-430d-afdc-a0d5b9cde659 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:52.476391 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03dbd7c-29b5-430d-afdc-a0d5b9cde659 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:52.476841 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03dbd7c-29b5-430d-afdc-a0d5b9cde659 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:52.484608 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:52.493580 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03dbd7c-29b5-430d-afdc-a0d5b9cde659 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 09:05:52.502261 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-e03dbd7c-29b5-430d-afdc-a0d5b9cde659 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 14 09:05:52.502261 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:05:52.502261 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:05:52.502261 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:05:52.502261 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:05:52.502261 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:05:52.502261 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:05:52.502261 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:05:52.502261 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:05:52.502261 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:05:52.502261 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:05:52.502261 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:05:52.502261 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 09:05:52.502261 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 09:05:52.502261 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 09:05:52.502261 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 09:05:52.503464 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 09:05:52.503464 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 09:05:52.503464 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 09:05:52.503464 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 09:05:52.503464 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:05:52.503464 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:05:52.503464 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:05:52.503464 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:05:52.503464 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:05:52.503464 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:05:52.503464 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:05:52.503464 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 14 09:05:52.503464 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:05:52.503464 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:05:52.503464 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:05:52.503464 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:05:52.504385 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:05:52.504385 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:05:52.504385 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:05:52.504385 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:05:52.504385 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 09:05:52.504385 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 09:05:52.504385 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 09:05:52.504385 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 09:05:52.504385 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 09:05:52.504385 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 09:05:52.504385 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 14 09:05:52.504385 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 14 09:05:52.508738 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 526/1063] 10.222.0.26 () {56 vars in 1120 bytes} [Thu Jul 14 09:05:52 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:05:52.509470 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03dbd7c-29b5-430d-afdc-a0d5b9cde659 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:52.519649 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 538/1064] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:05:52 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 09:05:52.541187 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908efebf-cc01-464f-8583-64e8b7d4cc24 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:52.570959 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:52.616823 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03dbd7c-29b5-430d-afdc-a0d5b9cde659 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:52.652914 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908efebf-cc01-464f-8583-64e8b7d4cc24 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:52.654206 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b535596c-4963-4cf2-aa01-a3110d466da7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:52.654892 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b535596c-4963-4cf2-aa01-a3110d466da7 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:52.655175 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b535596c-4963-4cf2-aa01-a3110d466da7 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:52.655483 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b535596c-4963-4cf2-aa01-a3110d466da7 None admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:52.656249 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b535596c-4963-4cf2-aa01-a3110d466da7 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:05:52.656560 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b535596c-4963-4cf2-aa01-a3110d466da7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:52.657970 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b535596c-4963-4cf2-aa01-a3110d466da7 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:52.671924 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 527/1065] 10.222.0.26 () {56 vars in 1115 bytes} [Thu Jul 14 09:05:52 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:05:52.673046 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b535596c-4963-4cf2-aa01-a3110d466da7 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:52.694523 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7e95ba-655a-4a5f-9478-4db097a57ab4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:52.695201 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7e95ba-655a-4a5f-9478-4db097a57ab4 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:52.695499 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7e95ba-655a-4a5f-9478-4db097a57ab4 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:52.695776 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7e95ba-655a-4a5f-9478-4db097a57ab4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:52.740351 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:52.769160 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7e95ba-655a-4a5f-9478-4db097a57ab4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:52.770624 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7e95ba-655a-4a5f-9478-4db097a57ab4 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:52.779399 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b535596c-4963-4cf2-aa01-a3110d466da7 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:52.802600 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:52.819442 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c13b8d6-aa8b-473f-b7a4-579e938c0ff1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:52.820313 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c13b8d6-aa8b-473f-b7a4-579e938c0ff1 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:52.820517 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c13b8d6-aa8b-473f-b7a4-579e938c0ff1 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:52.820794 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c13b8d6-aa8b-473f-b7a4-579e938c0ff1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:52.831606 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c13b8d6-aa8b-473f-b7a4-579e938c0ff1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:52.833113 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c13b8d6-aa8b-473f-b7a4-579e938c0ff1 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:52.850065 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-3c13b8d6-aa8b-473f-b7a4-579e938c0ff1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 09:05:52.850065 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:05:52.850065 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:05:52.850065 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:05:52.850065 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:05:52.850065 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:05:52.850065 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:05:52.850065 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:05:52.850065 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:05:52.850065 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:05:52.850065 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:05:52.850065 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:05:52.850065 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 09:05:52.850065 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 09:05:52.850065 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 09:05:52.851169 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 09:05:52.851169 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:05:52.851169 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:05:52.851169 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:05:52.851169 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:05:52.851169 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:05:52.851169 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:05:52.851169 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:05:52.851169 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 14 09:05:52.851169 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:05:52.851169 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:05:52.851169 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:05:52.851169 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:05:52.851169 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:05:52.851169 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:05:52.851169 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:05:52.851169 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:05:52.852485 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 09:05:52.852485 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 09:05:52.852485 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 09:05:52.852485 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 14 09:05:52.853703 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 528/1066] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jul 14 09:05:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 09:05:52.859161 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c13b8d6-aa8b-473f-b7a4-579e938c0ff1 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:52.883412 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 539/1067] 10.222.0.26 () {58 vars in 1282 bytes} [Thu Jul 14 09:05:52 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 14 09:05:52.888873 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7e95ba-655a-4a5f-9478-4db097a57ab4 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:52.966668 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:52.970695 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c13b8d6-aa8b-473f-b7a4-579e938c0ff1 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:52.993498 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7e95ba-655a-4a5f-9478-4db097a57ab4 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:53.007140 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f63e815-5615-48c7-8c4c-6f4805691c6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:53.008183 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f63e815-5615-48c7-8c4c-6f4805691c6e None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:53.008605 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f63e815-5615-48c7-8c4c-6f4805691c6e None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:53.008880 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f63e815-5615-48c7-8c4c-6f4805691c6e None admin] PATH_INFO: `/v3/domains` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:53.009689 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f63e815-5615-48c7-8c4c-6f4805691c6e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:05:53.009997 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f63e815-5615-48c7-8c4c-6f4805691c6e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:53.011387 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f63e815-5615-48c7-8c4c-6f4805691c6e None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:53.023728 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 529/1068] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Jul 14 09:05:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 09:05:53.028270 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5647fd12-5600-46a4-a176-d064dcafdd54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:53.029529 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5647fd12-5600-46a4-a176-d064dcafdd54 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:53.029983 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5647fd12-5600-46a4-a176-d064dcafdd54 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:53.030108 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f63e815-5615-48c7-8c4c-6f4805691c6e None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:53.030511 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5647fd12-5600-46a4-a176-d064dcafdd54 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:53.041111 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5647fd12-5600-46a4-a176-d064dcafdd54 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:53.042735 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5647fd12-5600-46a4-a176-d064dcafdd54 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:53.062495 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-5647fd12-5600-46a4-a176-d064dcafdd54 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 09:05:53.062495 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:05:53.062495 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:05:53.062495 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:05:53.062495 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:05:53.062495 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:05:53.062495 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:05:53.062495 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:05:53.062495 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:05:53.062495 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:05:53.062495 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:05:53.062495 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:05:53.062495 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 09:05:53.062495 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 09:05:53.062495 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 09:05:53.063550 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 09:05:53.063550 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:05:53.063550 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:05:53.063550 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:05:53.063550 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:05:53.063550 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:05:53.063550 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:05:53.063550 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:05:53.063550 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 14 09:05:53.063550 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:05:53.063550 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:05:53.063550 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:05:53.063550 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:05:53.063550 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:05:53.063550 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:05:53.063550 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:05:53.063550 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:05:53.064467 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 09:05:53.064467 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 09:05:53.064467 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 09:05:53.064467 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 14 09:05:53.064694 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:53.066620 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 540/1069] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jul 14 09:05:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 09:05:53.072702 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5647fd12-5600-46a4-a176-d064dcafdd54 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:53.137337 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f63e815-5615-48c7-8c4c-6f4805691c6e None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:53.175086 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5647fd12-5600-46a4-a176-d064dcafdd54 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:53.175343 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc947579-63e7-4bc6-9c1d-dc1d2cda3725 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:53.176216 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc947579-63e7-4bc6-9c1d-dc1d2cda3725 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:53.176578 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc947579-63e7-4bc6-9c1d-dc1d2cda3725 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:53.176858 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc947579-63e7-4bc6-9c1d-dc1d2cda3725 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:53.184963 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc947579-63e7-4bc6-9c1d-dc1d2cda3725 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:53.186444 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc947579-63e7-4bc6-9c1d-dc1d2cda3725 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:53.202093 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-dc947579-63e7-4bc6-9c1d-dc1d2cda3725 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 14 09:05:53.202093 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:05:53.202093 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:05:53.202093 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:05:53.202093 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:05:53.202093 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:05:53.202093 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:05:53.202093 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:05:53.202093 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:05:53.202093 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:05:53.202093 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:05:53.202093 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:05:53.202093 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 09:05:53.202093 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 09:05:53.202093 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 09:05:53.202093 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 09:05:53.203128 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:05:53.203128 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:05:53.203128 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 09:05:53.203128 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 09:05:53.203128 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 09:05:53.203128 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 09:05:53.203128 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:05:53.203128 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:05:53.203128 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:05:53.203128 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:05:53.203128 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:05:53.203128 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 14 09:05:53.203128 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:05:53.203128 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:05:53.203128 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:05:53.203128 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:05:53.203128 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:05:53.203128 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:05:53.204196 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:05:53.204196 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:05:53.204196 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 09:05:53.204196 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 09:05:53.204196 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 09:05:53.204196 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 09:05:53.204196 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 09:05:53.204196 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 09:05:53.204196 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 14 09:05:53.204196 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 14 09:05:53.206920 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:53.207340 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 530/1070] 10.222.0.26 () {56 vars in 1093 bytes} [Thu Jul 14 09:05:53 2022] GET /identity/v3/users/nova => generated 82 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 09:05:53.209579 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc947579-63e7-4bc6-9c1d-dc1d2cda3725 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:53.219858 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6bfde5-6a9b-46ed-b81c-1421162e39f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:53.220993 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6bfde5-6a9b-46ed-b81c-1421162e39f8 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:53.221417 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6bfde5-6a9b-46ed-b81c-1421162e39f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:53.221805 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6bfde5-6a9b-46ed-b81c-1421162e39f8 None admin] PATH_INFO: `/v3/domains` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:53.223000 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6bfde5-6a9b-46ed-b81c-1421162e39f8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:05:53.223373 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6bfde5-6a9b-46ed-b81c-1421162e39f8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:53.225096 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6bfde5-6a9b-46ed-b81c-1421162e39f8 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:53.247310 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 541/1071] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Jul 14 09:05:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 09:05:53.322974 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:53.333998 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc947579-63e7-4bc6-9c1d-dc1d2cda3725 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:53.372420 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd40b84-9726-4d15-8cb5-bb5a61332f80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:53.373101 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd40b84-9726-4d15-8cb5-bb5a61332f80 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:53.373382 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd40b84-9726-4d15-8cb5-bb5a61332f80 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:53.373658 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd40b84-9726-4d15-8cb5-bb5a61332f80 None admin] PATH_INFO: `/v3/users` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:53.374509 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd40b84-9726-4d15-8cb5-bb5a61332f80 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:05:53.374821 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd40b84-9726-4d15-8cb5-bb5a61332f80 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:53.376308 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd40b84-9726-4d15-8cb5-bb5a61332f80 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:53.395853 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 531/1072] 10.222.0.26 () {56 vars in 1133 bytes} [Thu Jul 14 09:05:53 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:05:53.401774 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6bfde5-6a9b-46ed-b81c-1421162e39f8 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:53.510628 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6bfde5-6a9b-46ed-b81c-1421162e39f8 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:53.549739 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a49dc45-ee73-4125-adbf-41743e50e710 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:53.550540 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a49dc45-ee73-4125-adbf-41743e50e710 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:53.550816 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a49dc45-ee73-4125-adbf-41743e50e710 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:53.551083 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a49dc45-ee73-4125-adbf-41743e50e710 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:53.561858 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a49dc45-ee73-4125-adbf-41743e50e710 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:53.563314 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a49dc45-ee73-4125-adbf-41743e50e710 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:53.578958 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f88744b-154a-43ab-a1a9-d5d7b5878f4b None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:53.579275 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f88744b-154a-43ab-a1a9-d5d7b5878f4b None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:53.579551 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f88744b-154a-43ab-a1a9-d5d7b5878f4b None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:53.579734 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-8a49dc45-ee73-4125-adbf-41743e50e710 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 09:05:53.579734 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:05:53.579734 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:05:53.579734 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:05:53.579734 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:05:53.579734 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:05:53.579734 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:05:53.579734 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:05:53.579734 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:05:53.579734 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:05:53.579734 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:05:53.579734 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:05:53.579734 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 09:05:53.579734 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 09:05:53.579734 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 09:05:53.580741 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 09:05:53.580741 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:05:53.580741 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:05:53.580741 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:05:53.580741 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:05:53.580741 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:05:53.580741 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:05:53.580741 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:05:53.580741 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 14 09:05:53.580741 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:05:53.580741 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:05:53.580741 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:05:53.580741 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:05:53.580741 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:05:53.580741 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:05:53.580741 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:05:53.580741 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:05:53.585458 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 09:05:53.585458 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 09:05:53.585458 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 09:05:53.585458 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 14 09:05:53.585747 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 532/1073] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:05:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:05:53.585839 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 542/1074] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jul 14 09:05:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:05:53.585928 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f3fb55-f9f1-4065-90fd-aa9b44e15eea None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:53.586122 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f3fb55-f9f1-4065-90fd-aa9b44e15eea None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:53.586402 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f3fb55-f9f1-4065-90fd-aa9b44e15eea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:53.588183 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a49dc45-ee73-4125-adbf-41743e50e710 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:53.602952 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-c9f3fb55-f9f1-4065-90fd-aa9b44e15eea None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:05:53.604156 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-c9f3fb55-f9f1-4065-90fd-aa9b44e15eea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:05:53.628805 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:53.672500 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 533/1075] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:05:53 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 09:05:53.689414 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a49dc45-ee73-4125-adbf-41743e50e710 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:53.694486 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f3fb55-f9f1-4065-90fd-aa9b44e15eea None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:53.724203 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a30483-2208-43ff-a856-e0420af87be2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:53.724869 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a30483-2208-43ff-a856-e0420af87be2 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:53.725158 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a30483-2208-43ff-a856-e0420af87be2 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:53.725423 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a30483-2208-43ff-a856-e0420af87be2 None admin] PATH_INFO: `/v3/domains` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:53.726148 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a30483-2208-43ff-a856-e0420af87be2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:05:53.726469 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a30483-2208-43ff-a856-e0420af87be2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:53.727869 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a30483-2208-43ff-a856-e0420af87be2 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:53.744282 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 543/1076] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Jul 14 09:05:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 09:05:53.745303 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a30483-2208-43ff-a856-e0420af87be2 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:53.773236 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:53.792393 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f3fb55-f9f1-4065-90fd-aa9b44e15eea None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:53.833146 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b33da28-f94d-46b4-8606-ad1517ef4455 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:53.833944 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b33da28-f94d-46b4-8606-ad1517ef4455 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:53.834236 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b33da28-f94d-46b4-8606-ad1517ef4455 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:53.834516 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b33da28-f94d-46b4-8606-ad1517ef4455 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:53.842962 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:53.847174 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b33da28-f94d-46b4-8606-ad1517ef4455 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 09:05:53.848063 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a30483-2208-43ff-a856-e0420af87be2 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:53.854681 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-4b33da28-f94d-46b4-8606-ad1517ef4455 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 09:05:53.854681 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:05:53.854681 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:05:53.854681 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:05:53.854681 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:05:53.854681 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:05:53.854681 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:05:53.854681 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:05:53.854681 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:05:53.854681 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:05:53.854681 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:05:53.854681 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:05:53.854681 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 09:05:53.854681 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 09:05:53.854681 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 09:05:53.854681 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 09:05:53.855880 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 09:05:53.855880 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 09:05:53.855880 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 09:05:53.855880 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 09:05:53.855880 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:05:53.855880 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:05:53.855880 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:05:53.855880 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:05:53.855880 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:05:53.855880 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:05:53.855880 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:05:53.855880 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 14 09:05:53.855880 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:05:53.855880 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:05:53.855880 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:05:53.855880 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:05:53.856968 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:05:53.856968 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:05:53.856968 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:05:53.856968 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:05:53.856968 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 09:05:53.856968 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 09:05:53.856968 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 09:05:53.856968 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 09:05:53.856968 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 09:05:53.856968 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 09:05:53.856968 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 09:05:53.856968 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 14 09:05:53.856968 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 534/1077] 10.222.0.26 () {56 vars in 1096 bytes} [Thu Jul 14 09:05:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 09:05:53.863361 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b33da28-f94d-46b4-8606-ad1517ef4455 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:53.883929 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92306cb1-2687-4537-860d-9aeffe86893c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:53.884739 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92306cb1-2687-4537-860d-9aeffe86893c None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:53.885019 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92306cb1-2687-4537-860d-9aeffe86893c None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:53.885295 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92306cb1-2687-4537-860d-9aeffe86893c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:53.892353 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92306cb1-2687-4537-860d-9aeffe86893c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:53.893812 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92306cb1-2687-4537-860d-9aeffe86893c None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:53.908793 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-92306cb1-2687-4537-860d-9aeffe86893c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 09:05:53.908793 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:05:53.908793 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:05:53.908793 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:05:53.908793 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:05:53.908793 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:05:53.908793 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:05:53.908793 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:05:53.908793 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:05:53.908793 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:05:53.908793 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:05:53.908793 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:05:53.908793 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 09:05:53.908793 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 09:05:53.908793 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 09:05:53.909904 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 09:05:53.909904 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:05:53.909904 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:05:53.909904 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:05:53.909904 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:05:53.909904 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:05:53.909904 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:05:53.909904 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:05:53.909904 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 14 09:05:53.909904 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:05:53.909904 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:05:53.909904 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:05:53.909904 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:05:53.909904 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:05:53.909904 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:05:53.909904 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:05:53.909904 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:05:53.911042 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 09:05:53.911042 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 09:05:53.911042 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 09:05:53.911042 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 09:05:53.911042 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 09:05:53.911042 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 09:05:53.911042 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 09:05:53.911042 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 14 09:05:53.911042 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 544/1078] 10.222.0.26 () {56 vars in 1111 bytes} [Thu Jul 14 09:05:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:05:53.915968 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92306cb1-2687-4537-860d-9aeffe86893c None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:53.966928 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b33da28-f94d-46b4-8606-ad1517ef4455 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:54.000366 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcfe9c0-21fd-487a-8180-ba2c7f87087f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:54.001039 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcfe9c0-21fd-487a-8180-ba2c7f87087f None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:54.001319 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcfe9c0-21fd-487a-8180-ba2c7f87087f None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:54.001593 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcfe9c0-21fd-487a-8180-ba2c7f87087f None admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:54.002339 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcfe9c0-21fd-487a-8180-ba2c7f87087f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:05:54.002640 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcfe9c0-21fd-487a-8180-ba2c7f87087f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:54.004044 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcfe9c0-21fd-487a-8180-ba2c7f87087f None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:54.010988 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92306cb1-2687-4537-860d-9aeffe86893c None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:54.011980 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 535/1079] 10.222.0.26 () {56 vars in 1099 bytes} [Thu Jul 14 09:05:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:05:54.017898 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcfe9c0-21fd-487a-8180-ba2c7f87087f None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:54.020498 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:54.045524 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbbefe9-43f9-44ad-9171-6db5ade167ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:54.046102 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbbefe9-43f9-44ad-9171-6db5ade167ff None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:54.046379 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbbefe9-43f9-44ad-9171-6db5ade167ff None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:54.046645 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbbefe9-43f9-44ad-9171-6db5ade167ff None admin] PATH_INFO: `/v3/projects` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:54.047445 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbbefe9-43f9-44ad-9171-6db5ade167ff None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:05:54.047748 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbbefe9-43f9-44ad-9171-6db5ade167ff None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:54.049168 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbbefe9-43f9-44ad-9171-6db5ade167ff None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:54.060367 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 545/1080] 10.222.0.26 () {56 vars in 1148 bytes} [Thu Jul 14 09:05:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 09:05:54.066236 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbbefe9-43f9-44ad-9171-6db5ade167ff None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:54.085160 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:54.119979 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcfe9c0-21fd-487a-8180-ba2c7f87087f None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:54.155523 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f099d65b-d046-496c-8762-d0e7dc4bb07b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:54.156686 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f099d65b-d046-496c-8762-d0e7dc4bb07b None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:54.156971 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f099d65b-d046-496c-8762-d0e7dc4bb07b None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:54.157242 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f099d65b-d046-496c-8762-d0e7dc4bb07b None admin] PATH_INFO: `/v3/users/02d4fb8036934bd9a5596addd9739d43` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:54.158418 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f099d65b-d046-496c-8762-d0e7dc4bb07b None admin] RBAC: Authorizing `identity:get_user(user_id=02d4fb8036934bd9a5596addd9739d43)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:54.159873 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f099d65b-d046-496c-8762-d0e7dc4bb07b None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:54.161504 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 536/1081] 10.222.0.26 () {56 vars in 1177 bytes} [Thu Jul 14 09:05:54 2022] GET /identity/v3/users/02d4fb8036934bd9a5596addd9739d43 => generated 285 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 09:05:54.164921 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbbefe9-43f9-44ad-9171-6db5ade167ff None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:54.167717 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f099d65b-d046-496c-8762-d0e7dc4bb07b None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:54.204503 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bd93fb-6058-45d0-8bfd-90fcd5e90d65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:54.205402 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bd93fb-6058-45d0-8bfd-90fcd5e90d65 None admin] REQUEST_METHOD: `PUT` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:54.205682 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bd93fb-6058-45d0-8bfd-90fcd5e90d65 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:54.205957 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bd93fb-6058-45d0-8bfd-90fcd5e90d65 None admin] PATH_INFO: `/v3/projects/33bf210d015847439655b9943d1b0261/users/538fff3c5a2b487eb44ae7c1a1198102/roles/70488f05098d4ce09e2fb03aaaea4566` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:54.208266 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bd93fb-6058-45d0-8bfd-90fcd5e90d65 None admin] RBAC: Authorizing `identity:create_grant(project_id=33bf210d015847439655b9943d1b0261, user_id=538fff3c5a2b487eb44ae7c1a1198102, role_id=70488f05098d4ce09e2fb03aaaea4566)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:54.209709 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bd93fb-6058-45d0-8bfd-90fcd5e90d65 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:54.223004 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 546/1082] 10.222.0.26 () {58 vars in 1439 bytes} [Thu Jul 14 09:05:54 2022] PUT /identity/v3/projects/33bf210d015847439655b9943d1b0261/users/538fff3c5a2b487eb44ae7c1a1198102/roles/70488f05098d4ce09e2fb03aaaea4566 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 09:05:54.235930 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:54.270616 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f099d65b-d046-496c-8762-d0e7dc4bb07b None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:54.308042 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3f4da5-6ea5-4456-8d63-cfbee5fcb7b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:54.308940 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3f4da5-6ea5-4456-8d63-cfbee5fcb7b5 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:54.309223 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3f4da5-6ea5-4456-8d63-cfbee5fcb7b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:54.309496 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3f4da5-6ea5-4456-8d63-cfbee5fcb7b5 None admin] PATH_INFO: `/v3/projects/d1b7367661964cf5a35f29aebdfa4650` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:54.310698 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3f4da5-6ea5-4456-8d63-cfbee5fcb7b5 None admin] RBAC: Authorizing `identity:get_project(project_id=d1b7367661964cf5a35f29aebdfa4650)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:54.312205 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3f4da5-6ea5-4456-8d63-cfbee5fcb7b5 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:54.314091 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 537/1083] 10.222.0.26 () {56 vars in 1186 bytes} [Thu Jul 14 09:05:54 2022] GET /identity/v3/projects/d1b7367661964cf5a35f29aebdfa4650 => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 09:05:54.320418 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bd93fb-6058-45d0-8bfd-90fcd5e90d65 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:54.406483 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:54.428874 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bd93fb-6058-45d0-8bfd-90fcd5e90d65 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:54.463031 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ba93ef-7235-40f5-9c7a-ab39434f2087 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:54.463951 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ba93ef-7235-40f5-9c7a-ab39434f2087 None admin] REQUEST_METHOD: `PUT` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:54.464226 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ba93ef-7235-40f5-9c7a-ab39434f2087 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:54.464611 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ba93ef-7235-40f5-9c7a-ab39434f2087 None admin] PATH_INFO: `/v3/projects/d1b7367661964cf5a35f29aebdfa4650/users/02d4fb8036934bd9a5596addd9739d43/roles/7b549d58d1724395b343609d895c80d0` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:54.466609 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ba93ef-7235-40f5-9c7a-ab39434f2087 None admin] RBAC: Authorizing `identity:create_grant(project_id=d1b7367661964cf5a35f29aebdfa4650, user_id=02d4fb8036934bd9a5596addd9739d43, role_id=7b549d58d1724395b343609d895c80d0)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:54.468071 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ba93ef-7235-40f5-9c7a-ab39434f2087 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:54.482086 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 547/1084] 10.222.0.26 () {58 vars in 1439 bytes} [Thu Jul 14 09:05:54 2022] PUT /identity/v3/projects/d1b7367661964cf5a35f29aebdfa4650/users/02d4fb8036934bd9a5596addd9739d43/roles/7b549d58d1724395b343609d895c80d0 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:05:56.210545 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1810f474-80a2-4cff-92f6-dd401697a010 None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:56.211094 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1810f474-80a2-4cff-92f6-dd401697a010 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:56.211094 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1810f474-80a2-4cff-92f6-dd401697a010 None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:56.212519 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 538/1085] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:05:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:05:56.217754 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0621b82f-086b-46ea-bc13-f0dd3d50385e None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:56.218149 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0621b82f-086b-46ea-bc13-f0dd3d50385e None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:56.218316 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0621b82f-086b-46ea-bc13-f0dd3d50385e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:56.235863 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-0621b82f-086b-46ea-bc13-f0dd3d50385e None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:05:56.237060 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-0621b82f-086b-46ea-bc13-f0dd3d50385e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:05:56.315627 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 548/1086] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:05:56 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 09:05:56.332953 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1810f474-80a2-4cff-92f6-dd401697a010 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:56.430783 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5534eb3a-ed9e-447c-bf03-90bb5179e2ab None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:56.431055 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5534eb3a-ed9e-447c-bf03-90bb5179e2ab None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:56.431367 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5534eb3a-ed9e-447c-bf03-90bb5179e2ab None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:56.432864 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 549/1087] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:05:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:05:56.437785 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ef44bd-0b48-4368-9f99-458054c0e012 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:56.438073 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ef44bd-0b48-4368-9f99-458054c0e012 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:56.438355 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ef44bd-0b48-4368-9f99-458054c0e012 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:56.441493 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1810f474-80a2-4cff-92f6-dd401697a010 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:56.455539 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-23ef44bd-0b48-4368-9f99-458054c0e012 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:05:56.456735 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-23ef44bd-0b48-4368-9f99-458054c0e012 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:05:56.476963 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5593e13b-d7ee-4381-9fe3-ecca9a175b86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:56.477772 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5593e13b-d7ee-4381-9fe3-ecca9a175b86 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:56.478074 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5593e13b-d7ee-4381-9fe3-ecca9a175b86 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:56.478355 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5593e13b-d7ee-4381-9fe3-ecca9a175b86 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:56.485532 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:56.489411 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5593e13b-d7ee-4381-9fe3-ecca9a175b86 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 09:05:56.498301 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-5593e13b-d7ee-4381-9fe3-ecca9a175b86 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 14 09:05:56.498301 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:05:56.498301 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:05:56.498301 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:05:56.498301 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:05:56.498301 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:05:56.498301 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:05:56.498301 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:05:56.498301 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:05:56.498301 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:05:56.498301 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:05:56.498301 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:05:56.498301 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 09:05:56.498301 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 09:05:56.498301 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 09:05:56.498301 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 09:05:56.499460 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 09:05:56.499460 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 09:05:56.499460 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 09:05:56.499460 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 09:05:56.499460 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:05:56.499460 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:05:56.499460 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:05:56.499460 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:05:56.499460 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:05:56.499460 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:05:56.499460 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:05:56.499460 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 14 09:05:56.499460 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:05:56.499460 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:05:56.499460 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:05:56.499460 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:05:56.500534 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:05:56.500534 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:05:56.500534 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:05:56.500534 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:05:56.500534 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 09:05:56.500534 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 09:05:56.500534 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 09:05:56.500534 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 09:05:56.500534 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 09:05:56.500534 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 09:05:56.500534 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 14 09:05:56.500534 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 14 09:05:56.500534 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 539/1088] 10.222.0.26 () {56 vars in 1120 bytes} [Thu Jul 14 09:05:56 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:05:56.505446 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5593e13b-d7ee-4381-9fe3-ecca9a175b86 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:56.522530 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:56.530657 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 550/1089] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:05:56 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 09:05:56.553995 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ef44bd-0b48-4368-9f99-458054c0e012 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:56.610770 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5593e13b-d7ee-4381-9fe3-ecca9a175b86 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:56.644929 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126b8204-fa80-447e-9f36-5bdca5622454 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:56.645592 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126b8204-fa80-447e-9f36-5bdca5622454 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:56.645880 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126b8204-fa80-447e-9f36-5bdca5622454 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:56.646157 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126b8204-fa80-447e-9f36-5bdca5622454 None admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:56.646906 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126b8204-fa80-447e-9f36-5bdca5622454 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:05:56.647218 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126b8204-fa80-447e-9f36-5bdca5622454 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:56.648616 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126b8204-fa80-447e-9f36-5bdca5622454 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:56.653005 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ef44bd-0b48-4368-9f99-458054c0e012 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:56.657105 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 540/1090] 10.222.0.26 () {56 vars in 1115 bytes} [Thu Jul 14 09:05:56 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 09:05:56.663169 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126b8204-fa80-447e-9f36-5bdca5622454 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:56.688450 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c53a431-9623-4b01-a234-b0f13be7a415 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:56.689256 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c53a431-9623-4b01-a234-b0f13be7a415 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:56.689542 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c53a431-9623-4b01-a234-b0f13be7a415 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:56.689818 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c53a431-9623-4b01-a234-b0f13be7a415 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:56.699909 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c53a431-9623-4b01-a234-b0f13be7a415 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 09:05:56.707529 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-7c53a431-9623-4b01-a234-b0f13be7a415 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 09:05:56.707529 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:05:56.707529 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:05:56.707529 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:05:56.707529 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:05:56.707529 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:05:56.707529 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:05:56.707529 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:05:56.707529 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:05:56.707529 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:05:56.707529 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:05:56.707529 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:05:56.707529 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 09:05:56.707529 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 09:05:56.707529 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 09:05:56.707529 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 09:05:56.708528 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 09:05:56.708528 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 09:05:56.708528 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 09:05:56.708528 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 09:05:56.708528 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:05:56.708528 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:05:56.708528 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:05:56.708528 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:05:56.708528 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:05:56.708528 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:05:56.708528 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:05:56.708528 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 14 09:05:56.708528 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:05:56.708528 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:05:56.708528 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:05:56.708528 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:05:56.709385 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:05:56.709385 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:05:56.709385 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:05:56.709385 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:05:56.709385 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 09:05:56.709385 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 09:05:56.709385 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 09:05:56.709385 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 09:05:56.709385 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 09:05:56.709385 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 09:05:56.709385 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 09:05:56.709385 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 14 09:05:56.709385 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 551/1091] 10.222.0.26 () {56 vars in 1096 bytes} [Thu Jul 14 09:05:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:05:56.715007 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c53a431-9623-4b01-a234-b0f13be7a415 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:56.723882 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:56.764440 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126b8204-fa80-447e-9f36-5bdca5622454 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:56.771353 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:56.801208 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f094a60-21d4-4206-bbc6-bced22f8d17f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:56.801997 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f094a60-21d4-4206-bbc6-bced22f8d17f None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:56.802277 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f094a60-21d4-4206-bbc6-bced22f8d17f None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:56.802550 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f094a60-21d4-4206-bbc6-bced22f8d17f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:56.813072 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f094a60-21d4-4206-bbc6-bced22f8d17f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:56.814507 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f094a60-21d4-4206-bbc6-bced22f8d17f None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:56.814975 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c53a431-9623-4b01-a234-b0f13be7a415 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:56.831324 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-1f094a60-21d4-4206-bbc6-bced22f8d17f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 09:05:56.831324 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:05:56.831324 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:05:56.831324 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:05:56.831324 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:05:56.831324 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:05:56.831324 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:05:56.831324 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:05:56.831324 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:05:56.831324 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:05:56.831324 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:05:56.831324 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:05:56.831324 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 09:05:56.831324 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 09:05:56.831324 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 09:05:56.832444 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 09:05:56.832444 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:05:56.832444 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:05:56.832444 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:05:56.832444 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:05:56.832444 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:05:56.832444 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:05:56.832444 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:05:56.832444 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 14 09:05:56.832444 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:05:56.832444 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:05:56.832444 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:05:56.832444 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:05:56.832444 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:05:56.832444 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:05:56.832444 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:05:56.832444 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:05:56.833429 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 09:05:56.833429 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 09:05:56.833429 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 09:05:56.833429 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 14 09:05:56.835342 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 541/1092] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jul 14 09:05:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 09:05:56.839036 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f094a60-21d4-4206-bbc6-bced22f8d17f None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:56.856121 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0592112f-ba79-4c3b-8b89-0f55c772f237 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:56.856797 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0592112f-ba79-4c3b-8b89-0f55c772f237 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:56.857082 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0592112f-ba79-4c3b-8b89-0f55c772f237 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:56.857361 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0592112f-ba79-4c3b-8b89-0f55c772f237 None admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:56.858117 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0592112f-ba79-4c3b-8b89-0f55c772f237 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:05:56.858424 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0592112f-ba79-4c3b-8b89-0f55c772f237 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:56.859838 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0592112f-ba79-4c3b-8b89-0f55c772f237 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:56.867868 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 552/1093] 10.222.0.26 () {56 vars in 1099 bytes} [Thu Jul 14 09:05:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 09:05:56.873486 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0592112f-ba79-4c3b-8b89-0f55c772f237 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:56.939977 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f094a60-21d4-4206-bbc6-bced22f8d17f None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:56.953045 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:56.971889 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0592112f-ba79-4c3b-8b89-0f55c772f237 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:56.974138 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09d4ec1-fdce-4417-9e42-48932656ac28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:56.974799 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09d4ec1-fdce-4417-9e42-48932656ac28 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:56.975083 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09d4ec1-fdce-4417-9e42-48932656ac28 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:56.975390 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09d4ec1-fdce-4417-9e42-48932656ac28 None admin] PATH_INFO: `/v3/domains` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:56.976114 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09d4ec1-fdce-4417-9e42-48932656ac28 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:05:56.976422 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09d4ec1-fdce-4417-9e42-48932656ac28 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:56.977793 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09d4ec1-fdce-4417-9e42-48932656ac28 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:56.989571 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 542/1094] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Jul 14 09:05:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 09:05:56.991611 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:56.995278 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09d4ec1-fdce-4417-9e42-48932656ac28 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:57.006976 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc98cfc6-a567-4f13-8215-6373b17573c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:57.007908 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc98cfc6-a567-4f13-8215-6373b17573c8 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:57.008195 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc98cfc6-a567-4f13-8215-6373b17573c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:57.008467 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc98cfc6-a567-4f13-8215-6373b17573c8 None admin] PATH_INFO: `/v3/users/02d4fb8036934bd9a5596addd9739d43` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:57.009695 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc98cfc6-a567-4f13-8215-6373b17573c8 None admin] RBAC: Authorizing `identity:get_user(user_id=02d4fb8036934bd9a5596addd9739d43)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:57.011234 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc98cfc6-a567-4f13-8215-6373b17573c8 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:57.017572 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 553/1095] 10.222.0.26 () {56 vars in 1177 bytes} [Thu Jul 14 09:05:56 2022] GET /identity/v3/users/02d4fb8036934bd9a5596addd9739d43 => generated 285 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 09:05:57.018362 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc98cfc6-a567-4f13-8215-6373b17573c8 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:57.093835 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09d4ec1-fdce-4417-9e42-48932656ac28 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:57.116964 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc98cfc6-a567-4f13-8215-6373b17573c8 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:57.127975 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49d6925-62e8-48b1-b4c3-302fa82cab67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:57.128770 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49d6925-62e8-48b1-b4c3-302fa82cab67 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:57.129058 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49d6925-62e8-48b1-b4c3-302fa82cab67 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:57.129336 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49d6925-62e8-48b1-b4c3-302fa82cab67 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:57.136458 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49d6925-62e8-48b1-b4c3-302fa82cab67 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:57.137895 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49d6925-62e8-48b1-b4c3-302fa82cab67 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:57.151982 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1965332f-0816-4379-8dd3-9c369d507406 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:57.152577 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-a49d6925-62e8-48b1-b4c3-302fa82cab67 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 14 09:05:57.152577 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:05:57.152577 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:05:57.152577 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:05:57.152577 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:05:57.152577 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:05:57.152577 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:05:57.152577 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:05:57.152577 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:05:57.152577 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:05:57.152577 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:05:57.152577 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:05:57.152577 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 09:05:57.152577 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 09:05:57.152577 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 09:05:57.152577 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 09:05:57.153613 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:05:57.153613 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:05:57.153613 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 09:05:57.153613 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 09:05:57.153613 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 09:05:57.153613 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 09:05:57.153613 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:05:57.153613 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:05:57.153613 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:05:57.153613 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:05:57.153613 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:05:57.153613 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 14 09:05:57.153613 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:05:57.153613 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:05:57.153613 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:05:57.153613 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:05:57.153613 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:05:57.153613 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:05:57.154681 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:05:57.154681 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:05:57.154681 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 09:05:57.154681 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 09:05:57.154681 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 09:05:57.154681 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 09:05:57.154681 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 09:05:57.154681 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 09:05:57.154681 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 14 09:05:57.154681 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 14 09:05:57.155269 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1965332f-0816-4379-8dd3-9c369d507406 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:57.155269 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1965332f-0816-4379-8dd3-9c369d507406 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:57.155269 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1965332f-0816-4379-8dd3-9c369d507406 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:57.155475 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 543/1096] 10.222.0.26 () {56 vars in 1093 bytes} [Thu Jul 14 09:05:56 2022] GET /identity/v3/users/nova => generated 82 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 09:05:57.159623 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49d6925-62e8-48b1-b4c3-302fa82cab67 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:57.189763 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:57.209546 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:57.222288 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1965332f-0816-4379-8dd3-9c369d507406 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:57.223778 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1965332f-0816-4379-8dd3-9c369d507406 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:57.259167 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49d6925-62e8-48b1-b4c3-302fa82cab67 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:57.294729 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fd2166-d435-437b-b3d4-c9af6d554f07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:57.295369 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fd2166-d435-437b-b3d4-c9af6d554f07 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:57.295656 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fd2166-d435-437b-b3d4-c9af6d554f07 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:57.295930 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fd2166-d435-437b-b3d4-c9af6d554f07 None admin] PATH_INFO: `/v3/users` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:57.296771 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fd2166-d435-437b-b3d4-c9af6d554f07 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:05:57.297082 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fd2166-d435-437b-b3d4-c9af6d554f07 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:57.299073 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fd2166-d435-437b-b3d4-c9af6d554f07 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:57.317196 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 544/1097] 10.222.0.26 () {56 vars in 1133 bytes} [Thu Jul 14 09:05:57 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 09:05:57.323326 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fd2166-d435-437b-b3d4-c9af6d554f07 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:57.326254 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 554/1098] 10.222.0.26 () {58 vars in 1282 bytes} [Thu Jul 14 09:05:57 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 14 09:05:57.332015 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1965332f-0816-4379-8dd3-9c369d507406 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:57.414047 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:57.416674 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:57.424512 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fd2166-d435-437b-b3d4-c9af6d554f07 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:57.430335 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1965332f-0816-4379-8dd3-9c369d507406 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:57.458410 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802dd6b2-6339-42d6-bb1a-31d3357857f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:57.459038 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802dd6b2-6339-42d6-bb1a-31d3357857f2 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:57.459350 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802dd6b2-6339-42d6-bb1a-31d3357857f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:57.459619 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802dd6b2-6339-42d6-bb1a-31d3357857f2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:57.464656 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409b9952-940d-488d-96ba-c82b0f3939aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:57.465521 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409b9952-940d-488d-96ba-c82b0f3939aa None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:57.465805 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409b9952-940d-488d-96ba-c82b0f3939aa None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:57.466080 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409b9952-940d-488d-96ba-c82b0f3939aa None admin] PATH_INFO: `/v3/projects/d1b7367661964cf5a35f29aebdfa4650` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:57.467349 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409b9952-940d-488d-96ba-c82b0f3939aa None admin] RBAC: Authorizing `identity:get_project(project_id=d1b7367661964cf5a35f29aebdfa4650)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:57.468801 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409b9952-940d-488d-96ba-c82b0f3939aa None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:57.470530 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 555/1099] 10.222.0.26 () {56 vars in 1186 bytes} [Thu Jul 14 09:05:57 2022] GET /identity/v3/projects/d1b7367661964cf5a35f29aebdfa4650 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:05:57.476579 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409b9952-940d-488d-96ba-c82b0f3939aa None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:57.530606 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802dd6b2-6339-42d6-bb1a-31d3357857f2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:57.532236 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802dd6b2-6339-42d6-bb1a-31d3357857f2 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:57.574925 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409b9952-940d-488d-96ba-c82b0f3939aa None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:57.607758 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b1d5e6-6b25-46b0-9da5-ce38cd3131f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:57.608410 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b1d5e6-6b25-46b0-9da5-ce38cd3131f2 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:57.608694 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b1d5e6-6b25-46b0-9da5-ce38cd3131f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:57.608971 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b1d5e6-6b25-46b0-9da5-ce38cd3131f2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:57.609804 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b1d5e6-6b25-46b0-9da5-ce38cd3131f2 None admin] RBAC: Adding query filter params (role.id=7b549d58d1724395b343609d895c80d0, scope.project.id=d1b7367661964cf5a35f29aebdfa4650, user.id=02d4fb8036934bd9a5596addd9739d43) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:05:57.610114 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b1d5e6-6b25-46b0-9da5-ce38cd3131f2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:57.611573 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b1d5e6-6b25-46b0-9da5-ce38cd3131f2 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:57.617243 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 556/1100] 10.222.0.26 () {56 vars in 1374 bytes} [Thu Jul 14 09:05:57 2022] GET /identity/v3/role_assignments?user.id=02d4fb8036934bd9a5596addd9739d43&scope.project.id=d1b7367661964cf5a35f29aebdfa4650&role.id=7b549d58d1724395b343609d895c80d0 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 09:05:57.620765 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:57.631784 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 545/1101] 10.222.0.26 () {58 vars in 1282 bytes} [Thu Jul 14 09:05:57 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 09:05:57.638042 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b1d5e6-6b25-46b0-9da5-ce38cd3131f2 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:57.662130 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:57.747669 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b1d5e6-6b25-46b0-9da5-ce38cd3131f2 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:57.785870 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156c11aa-f1b0-4565-b50f-61b90355e20d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:57.786684 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156c11aa-f1b0-4565-b50f-61b90355e20d None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:57.786968 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156c11aa-f1b0-4565-b50f-61b90355e20d None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:57.787379 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156c11aa-f1b0-4565-b50f-61b90355e20d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:57.798521 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156c11aa-f1b0-4565-b50f-61b90355e20d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:57.800028 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156c11aa-f1b0-4565-b50f-61b90355e20d None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:57.817733 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-156c11aa-f1b0-4565-b50f-61b90355e20d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 09:05:57.817733 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:05:57.817733 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:05:57.817733 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:05:57.817733 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:05:57.817733 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:05:57.817733 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:05:57.817733 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:05:57.817733 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:05:57.817733 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:05:57.817733 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:05:57.817733 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:05:57.817733 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 09:05:57.817733 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 09:05:57.817733 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 09:05:57.818679 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 09:05:57.818679 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:05:57.818679 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:05:57.818679 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:05:57.818679 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:05:57.818679 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:05:57.818679 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:05:57.818679 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:05:57.818679 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 14 09:05:57.818679 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:05:57.818679 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:05:57.818679 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:05:57.818679 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:05:57.818679 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:05:57.818679 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:05:57.818679 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:05:57.818679 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:05:57.819700 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 09:05:57.819700 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 09:05:57.819700 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 09:05:57.819700 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 14 09:05:57.819700 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 557/1102] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jul 14 09:05:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:05:57.825018 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802dd6b2-6339-42d6-bb1a-31d3357857f2 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:57.923882 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802dd6b2-6339-42d6-bb1a-31d3357857f2 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:57.957994 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca71d628-23e4-406b-bbb7-7bd0a2967d14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:57.958661 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca71d628-23e4-406b-bbb7-7bd0a2967d14 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:57.958943 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca71d628-23e4-406b-bbb7-7bd0a2967d14 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:57.959210 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca71d628-23e4-406b-bbb7-7bd0a2967d14 None admin] PATH_INFO: `/v3/domains` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:57.959971 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca71d628-23e4-406b-bbb7-7bd0a2967d14 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:05:57.960278 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca71d628-23e4-406b-bbb7-7bd0a2967d14 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:57.961654 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca71d628-23e4-406b-bbb7-7bd0a2967d14 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:57.973401 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 546/1103] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Jul 14 09:05:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:05:57.979174 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156c11aa-f1b0-4565-b50f-61b90355e20d None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:58.053263 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:58.085373 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156c11aa-f1b0-4565-b50f-61b90355e20d None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:58.118874 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847b1bf2-239c-4b96-8327-c3749343eae7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:58.119728 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847b1bf2-239c-4b96-8327-c3749343eae7 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:58.120153 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847b1bf2-239c-4b96-8327-c3749343eae7 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:58.120404 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847b1bf2-239c-4b96-8327-c3749343eae7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:58.127059 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847b1bf2-239c-4b96-8327-c3749343eae7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:58.128563 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847b1bf2-239c-4b96-8327-c3749343eae7 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:58.142609 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-847b1bf2-239c-4b96-8327-c3749343eae7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 09:05:58.142609 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:05:58.142609 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:05:58.142609 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:05:58.142609 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:05:58.142609 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:05:58.142609 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:05:58.142609 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:05:58.142609 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:05:58.142609 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:05:58.142609 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:05:58.142609 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:05:58.142609 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 09:05:58.142609 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 09:05:58.142609 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 09:05:58.143599 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 09:05:58.143599 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:05:58.143599 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:05:58.143599 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:05:58.143599 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:05:58.143599 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:05:58.143599 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:05:58.143599 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:05:58.143599 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 14 09:05:58.143599 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:05:58.143599 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:05:58.143599 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:05:58.143599 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:05:58.143599 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:05:58.143599 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:05:58.143599 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:05:58.143599 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:05:58.144515 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 09:05:58.144515 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 09:05:58.144515 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 09:05:58.144515 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 09:05:58.144515 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 09:05:58.144515 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 09:05:58.144515 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 09:05:58.144515 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 14 09:05:58.144515 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 558/1104] 10.222.0.26 () {56 vars in 1111 bytes} [Thu Jul 14 09:05:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:05:58.149964 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca71d628-23e4-406b-bbb7-7bd0a2967d14 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:58.201562 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:58.252071 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca71d628-23e4-406b-bbb7-7bd0a2967d14 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:58.286645 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f433716-690d-4a94-bf5a-7dce0ccfcc09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:58.287287 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f433716-690d-4a94-bf5a-7dce0ccfcc09 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:58.287589 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f433716-690d-4a94-bf5a-7dce0ccfcc09 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:58.287862 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f433716-690d-4a94-bf5a-7dce0ccfcc09 None admin] PATH_INFO: `/v3/projects` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:58.288649 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f433716-690d-4a94-bf5a-7dce0ccfcc09 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:05:58.288952 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f433716-690d-4a94-bf5a-7dce0ccfcc09 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:58.290375 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f433716-690d-4a94-bf5a-7dce0ccfcc09 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:58.303058 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 547/1105] 10.222.0.26 () {56 vars in 1148 bytes} [Thu Jul 14 09:05:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:05:58.309401 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847b1bf2-239c-4b96-8327-c3749343eae7 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:58.408171 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847b1bf2-239c-4b96-8327-c3749343eae7 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:58.437888 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:58.443627 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f861ecc3-7a97-4481-958a-8791453c5b32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:58.444290 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f861ecc3-7a97-4481-958a-8791453c5b32 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:58.444577 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f861ecc3-7a97-4481-958a-8791453c5b32 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:58.444852 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f861ecc3-7a97-4481-958a-8791453c5b32 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:58.445695 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f861ecc3-7a97-4481-958a-8791453c5b32 None admin] RBAC: Adding query filter params (role.id=70488f05098d4ce09e2fb03aaaea4566, scope.project.id=33bf210d015847439655b9943d1b0261, user.id=538fff3c5a2b487eb44ae7c1a1198102) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:05:58.446005 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f861ecc3-7a97-4481-958a-8791453c5b32 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:58.447467 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f861ecc3-7a97-4481-958a-8791453c5b32 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:58.453348 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 559/1106] 10.222.0.26 () {56 vars in 1374 bytes} [Thu Jul 14 09:05:58 2022] GET /identity/v3/role_assignments?user.id=538fff3c5a2b487eb44ae7c1a1198102&scope.project.id=33bf210d015847439655b9943d1b0261&role.id=70488f05098d4ce09e2fb03aaaea4566 => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:05:59.643833 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471b9aa8-d816-4c51-bd81-ccfb2c7b95ae None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:59.644653 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471b9aa8-d816-4c51-bd81-ccfb2c7b95ae None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:59.644653 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471b9aa8-d816-4c51-bd81-ccfb2c7b95ae None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:59.646973 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 548/1107] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:05:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:05:59.653550 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e33fe5e-40d4-4bce-b5a1-2e81bf3364be None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:59.654229 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e33fe5e-40d4-4bce-b5a1-2e81bf3364be None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:59.654320 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e33fe5e-40d4-4bce-b5a1-2e81bf3364be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:59.682711 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-9e33fe5e-40d4-4bce-b5a1-2e81bf3364be None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:05:59.683920 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-9e33fe5e-40d4-4bce-b5a1-2e81bf3364be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:05:59.752748 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 560/1108] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:05:59 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 09:05:59.780616 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471b9aa8-d816-4c51-bd81-ccfb2c7b95ae None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:05:59.892187 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471b9aa8-d816-4c51-bd81-ccfb2c7b95ae None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:05:59.908532 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:05:59.936914 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a60a31d-3b6e-46f4-b9fb-995158b6f160 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:05:59.937698 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a60a31d-3b6e-46f4-b9fb-995158b6f160 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:05:59.937982 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a60a31d-3b6e-46f4-b9fb-995158b6f160 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:05:59.938257 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a60a31d-3b6e-46f4-b9fb-995158b6f160 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:05:59.939489 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a60a31d-3b6e-46f4-b9fb-995158b6f160 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:05:59.940926 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a60a31d-3b6e-46f4-b9fb-995158b6f160 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:05:59.942590 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 549/1109] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jul 14 09:05:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:05:59.948411 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e33fe5e-40d4-4bce-b5a1-2e81bf3364be None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:06:00.056142 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e33fe5e-40d4-4bce-b5a1-2e81bf3364be None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:06:00.082146 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:06:00.091492 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c16a398-9fd0-4133-b64f-1bb1f1b41680 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:06:00.092158 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c16a398-9fd0-4133-b64f-1bb1f1b41680 None admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:06:00.092437 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c16a398-9fd0-4133-b64f-1bb1f1b41680 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:06:00.092712 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c16a398-9fd0-4133-b64f-1bb1f1b41680 None admin] PATH_INFO: `/v3/users` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:06:00.093424 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c16a398-9fd0-4133-b64f-1bb1f1b41680 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:06:00.094858 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c16a398-9fd0-4133-b64f-1bb1f1b41680 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:06:00.115911 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 561/1110] 10.222.0.26 () {60 vars in 1132 bytes} [Thu Jul 14 09:05:59 2022] POST /identity/v3/users => generated 286 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 09:06:02.148836 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81608e34-7264-4af8-8557-ca4eca068de9 None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:06:02.149425 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81608e34-7264-4af8-8557-ca4eca068de9 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:06:02.149425 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81608e34-7264-4af8-8557-ca4eca068de9 None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:06:02.151135 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 550/1111] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:06:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:06:02.157493 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c60c7d1-22e0-4f1e-b05b-9532f0ba612e None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:06:02.158224 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c60c7d1-22e0-4f1e-b05b-9532f0ba612e None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:06:02.158361 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c60c7d1-22e0-4f1e-b05b-9532f0ba612e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:06:02.180124 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-8c60c7d1-22e0-4f1e-b05b-9532f0ba612e None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:06:02.181312 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-8c60c7d1-22e0-4f1e-b05b-9532f0ba612e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:06:02.254894 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 562/1112] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:06:02 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 09:06:02.280290 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81608e34-7264-4af8-8557-ca4eca068de9 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:06:02.395586 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81608e34-7264-4af8-8557-ca4eca068de9 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:06:02.432655 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a7dc46-4b63-4eb9-a24f-a0769c2a143f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:06:02.433437 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a7dc46-4b63-4eb9-a24f-a0769c2a143f None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:06:02.433716 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a7dc46-4b63-4eb9-a24f-a0769c2a143f None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:06:02.433987 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a7dc46-4b63-4eb9-a24f-a0769c2a143f None admin] PATH_INFO: `/v3/roles/c4bab4a02e5c4f5f97a024e844a0c072` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:06:02.435500 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a7dc46-4b63-4eb9-a24f-a0769c2a143f None admin] RBAC: Authorizing `identity:get_role(role_id=c4bab4a02e5c4f5f97a024e844a0c072)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:06:02.436877 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a7dc46-4b63-4eb9-a24f-a0769c2a143f None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:06:02.438245 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 551/1113] 10.222.0.26 () {56 vars in 1177 bytes} [Thu Jul 14 09:06:02 2022] GET /identity/v3/roles/c4bab4a02e5c4f5f97a024e844a0c072 => generated 223 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:06:02.447671 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c60c7d1-22e0-4f1e-b05b-9532f0ba612e None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:06:02.557285 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c60c7d1-22e0-4f1e-b05b-9532f0ba612e None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:06:02.570571 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:06:02.593090 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3eddca-d5c5-4ed3-b4ca-54a183231869 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:06:02.593973 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3eddca-d5c5-4ed3-b4ca-54a183231869 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:06:02.594260 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3eddca-d5c5-4ed3-b4ca-54a183231869 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:06:02.594559 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3eddca-d5c5-4ed3-b4ca-54a183231869 None admin] PATH_INFO: `/v3/users/e2e3d43660964d0aadaff82d24e41bcf` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:06:02.607785 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3eddca-d5c5-4ed3-b4ca-54a183231869 None admin] RBAC: Authorizing `identity:get_user(user_id=e2e3d43660964d0aadaff82d24e41bcf)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:06:02.609239 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3eddca-d5c5-4ed3-b4ca-54a183231869 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:06:02.610930 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 563/1114] 10.222.0.26 () {56 vars in 1177 bytes} [Thu Jul 14 09:06:02 2022] GET /identity/v3/users/e2e3d43660964d0aadaff82d24e41bcf => generated 286 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:06:02.617093 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a7dc46-4b63-4eb9-a24f-a0769c2a143f None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:06:02.669817 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:06:02.727124 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a7dc46-4b63-4eb9-a24f-a0769c2a143f None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:06:02.763314 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435621d2-5d57-4104-a974-4d2213f7dc2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:06:02.763954 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435621d2-5d57-4104-a974-4d2213f7dc2d None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:06:02.764243 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435621d2-5d57-4104-a974-4d2213f7dc2d None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:06:02.764516 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435621d2-5d57-4104-a974-4d2213f7dc2d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:06:02.833343 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435621d2-5d57-4104-a974-4d2213f7dc2d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:06:02.834781 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435621d2-5d57-4104-a974-4d2213f7dc2d None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:06:02.879639 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:06:02.939712 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 552/1115] 10.222.0.26 () {58 vars in 1282 bytes} [Thu Jul 14 09:06:02 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 09:06:02.946013 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3eddca-d5c5-4ed3-b4ca-54a183231869 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:06:03.050136 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3eddca-d5c5-4ed3-b4ca-54a183231869 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:06:03.088046 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91f1231-96b0-4601-a7a2-011ad6d79f75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:06:03.088895 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91f1231-96b0-4601-a7a2-011ad6d79f75 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:06:03.089173 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91f1231-96b0-4601-a7a2-011ad6d79f75 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:06:03.089443 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91f1231-96b0-4601-a7a2-011ad6d79f75 None admin] PATH_INFO: `/v3/projects/d1b7367661964cf5a35f29aebdfa4650` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:06:03.091226 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91f1231-96b0-4601-a7a2-011ad6d79f75 None admin] RBAC: Authorizing `identity:get_project(project_id=d1b7367661964cf5a35f29aebdfa4650)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:06:03.093893 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91f1231-96b0-4601-a7a2-011ad6d79f75 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:06:03.096698 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 564/1116] 10.222.0.26 () {56 vars in 1186 bytes} [Thu Jul 14 09:06:02 2022] GET /identity/v3/projects/d1b7367661964cf5a35f29aebdfa4650 => generated 311 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:06:03.105482 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435621d2-5d57-4104-a974-4d2213f7dc2d None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:06:03.219023 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435621d2-5d57-4104-a974-4d2213f7dc2d None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:06:03.256033 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b342a6-1c08-4f1b-87cc-8e7d60065130 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:06:03.256653 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b342a6-1c08-4f1b-87cc-8e7d60065130 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:06:03.256954 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b342a6-1c08-4f1b-87cc-8e7d60065130 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:06:03.257231 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b342a6-1c08-4f1b-87cc-8e7d60065130 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:06:03.258057 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b342a6-1c08-4f1b-87cc-8e7d60065130 None admin] RBAC: Adding query filter params (role.id=c4bab4a02e5c4f5f97a024e844a0c072, scope.project.id=d1b7367661964cf5a35f29aebdfa4650, user.id=e2e3d43660964d0aadaff82d24e41bcf) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:06:03.258371 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b342a6-1c08-4f1b-87cc-8e7d60065130 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:06:03.259838 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b342a6-1c08-4f1b-87cc-8e7d60065130 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:06:03.265719 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 553/1117] 10.222.0.26 () {56 vars in 1374 bytes} [Thu Jul 14 09:06:03 2022] GET /identity/v3/role_assignments?user.id=e2e3d43660964d0aadaff82d24e41bcf&scope.project.id=d1b7367661964cf5a35f29aebdfa4650&role.id=c4bab4a02e5c4f5f97a024e844a0c072 => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:06:05.250365 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6becd028-1539-49c5-9e94-acb40afd0722 None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:06:05.251259 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6becd028-1539-49c5-9e94-acb40afd0722 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:06:05.251259 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6becd028-1539-49c5-9e94-acb40afd0722 None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:06:05.253573 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 565/1118] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:06:05 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:06:05.260008 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215c8ac7-a1ad-46ee-a013-ede072a7856c None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:06:05.260429 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215c8ac7-a1ad-46ee-a013-ede072a7856c None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:06:05.260621 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215c8ac7-a1ad-46ee-a013-ede072a7856c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:06:05.287808 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-215c8ac7-a1ad-46ee-a013-ede072a7856c None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:06:05.289710 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-215c8ac7-a1ad-46ee-a013-ede072a7856c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:06:05.293104 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:06:05.362425 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 554/1119] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:06:05 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 09:06:05.388079 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6becd028-1539-49c5-9e94-acb40afd0722 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:06:05.436386 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:06:05.501861 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6becd028-1539-49c5-9e94-acb40afd0722 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:06:05.539180 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e420567c-c202-408c-8882-9238ec9cfced None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:06:05.540077 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e420567c-c202-408c-8882-9238ec9cfced None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:06:05.540352 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e420567c-c202-408c-8882-9238ec9cfced None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:06:05.540620 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e420567c-c202-408c-8882-9238ec9cfced None admin] PATH_INFO: `/v3/roles/c4bab4a02e5c4f5f97a024e844a0c072` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:06:05.542150 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e420567c-c202-408c-8882-9238ec9cfced None admin] RBAC: Authorizing `identity:get_role(role_id=c4bab4a02e5c4f5f97a024e844a0c072)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:06:05.543561 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e420567c-c202-408c-8882-9238ec9cfced None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:06:05.544961 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 566/1120] 10.222.0.26 () {56 vars in 1177 bytes} [Thu Jul 14 09:06:05 2022] GET /identity/v3/roles/c4bab4a02e5c4f5f97a024e844a0c072 => generated 223 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:06:05.551137 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215c8ac7-a1ad-46ee-a013-ede072a7856c None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:06:05.657602 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215c8ac7-a1ad-46ee-a013-ede072a7856c None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:06:05.693088 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5fc430-d347-42f2-898e-066d0dd29d64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:06:05.693948 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5fc430-d347-42f2-898e-066d0dd29d64 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:06:05.694230 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5fc430-d347-42f2-898e-066d0dd29d64 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:06:05.694505 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5fc430-d347-42f2-898e-066d0dd29d64 None admin] PATH_INFO: `/v3/users/e2e3d43660964d0aadaff82d24e41bcf` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:06:05.695757 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5fc430-d347-42f2-898e-066d0dd29d64 None admin] RBAC: Authorizing `identity:get_user(user_id=e2e3d43660964d0aadaff82d24e41bcf)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:06:05.697202 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5fc430-d347-42f2-898e-066d0dd29d64 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:06:05.698848 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 555/1121] 10.222.0.26 () {56 vars in 1177 bytes} [Thu Jul 14 09:06:05 2022] GET /identity/v3/users/e2e3d43660964d0aadaff82d24e41bcf => generated 286 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:06:05.704990 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e420567c-c202-408c-8882-9238ec9cfced None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:06:05.818572 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e420567c-c202-408c-8882-9238ec9cfced None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:06:05.825525 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:06:05.855513 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea39248-b93a-4afa-93a7-a02284f8493c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:06:05.856364 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea39248-b93a-4afa-93a7-a02284f8493c None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:06:05.856641 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea39248-b93a-4afa-93a7-a02284f8493c None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:06:05.856911 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea39248-b93a-4afa-93a7-a02284f8493c None admin] PATH_INFO: `/v3/projects/d1b7367661964cf5a35f29aebdfa4650` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:06:05.858136 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea39248-b93a-4afa-93a7-a02284f8493c None admin] RBAC: Authorizing `identity:get_project(project_id=d1b7367661964cf5a35f29aebdfa4650)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:06:05.859598 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea39248-b93a-4afa-93a7-a02284f8493c None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:06:05.861301 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 567/1122] 10.222.0.26 () {56 vars in 1186 bytes} [Thu Jul 14 09:06:05 2022] GET /identity/v3/projects/d1b7367661964cf5a35f29aebdfa4650 => generated 311 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:06:05.867668 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5fc430-d347-42f2-898e-066d0dd29d64 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:06:05.879889 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:06:05.976580 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5fc430-d347-42f2-898e-066d0dd29d64 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:06:06.011873 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11c24da-60e4-4b43-84e5-e1f044ffec1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:06:06.012736 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11c24da-60e4-4b43-84e5-e1f044ffec1d None admin] REQUEST_METHOD: `PUT` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:06:06.013011 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11c24da-60e4-4b43-84e5-e1f044ffec1d None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:06:06.013275 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11c24da-60e4-4b43-84e5-e1f044ffec1d None admin] PATH_INFO: `/v3/projects/d1b7367661964cf5a35f29aebdfa4650/users/e2e3d43660964d0aadaff82d24e41bcf/roles/c4bab4a02e5c4f5f97a024e844a0c072` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:06:06.016008 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11c24da-60e4-4b43-84e5-e1f044ffec1d None admin] RBAC: Authorizing `identity:create_grant(project_id=d1b7367661964cf5a35f29aebdfa4650, user_id=e2e3d43660964d0aadaff82d24e41bcf, role_id=c4bab4a02e5c4f5f97a024e844a0c072)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:06:06.017433 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11c24da-60e4-4b43-84e5-e1f044ffec1d None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:06:06.033652 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 556/1123] 10.222.0.26 () {58 vars in 1439 bytes} [Thu Jul 14 09:06:05 2022] PUT /identity/v3/projects/d1b7367661964cf5a35f29aebdfa4650/users/e2e3d43660964d0aadaff82d24e41bcf/roles/c4bab4a02e5c4f5f97a024e844a0c072 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:06:08.012719 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d625285-0d45-45ca-b263-209cab1bcbd5 None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:06:08.013301 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d625285-0d45-45ca-b263-209cab1bcbd5 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:06:08.013301 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d625285-0d45-45ca-b263-209cab1bcbd5 None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:06:08.014668 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 568/1124] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:06:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:06:08.021150 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6b6704-a69c-400e-b3a6-5cfc29ec7ec1 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:06:08.021870 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6b6704-a69c-400e-b3a6-5cfc29ec7ec1 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:06:08.022050 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6b6704-a69c-400e-b3a6-5cfc29ec7ec1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:06:08.050244 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-ea6b6704-a69c-400e-b3a6-5cfc29ec7ec1 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:06:08.051460 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-ea6b6704-a69c-400e-b3a6-5cfc29ec7ec1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:06:08.115751 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:06:08.123089 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 557/1125] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:06:08 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 09:06:08.146949 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d625285-0d45-45ca-b263-209cab1bcbd5 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:06:08.266682 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d625285-0d45-45ca-b263-209cab1bcbd5 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:06:08.302867 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a4ea10-0b87-489d-bc67-fb9bd6837b89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:06:08.303888 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a4ea10-0b87-489d-bc67-fb9bd6837b89 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:06:08.304169 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a4ea10-0b87-489d-bc67-fb9bd6837b89 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:06:08.304446 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a4ea10-0b87-489d-bc67-fb9bd6837b89 None admin] PATH_INFO: `/v3/roles/c4bab4a02e5c4f5f97a024e844a0c072` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:06:08.305945 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a4ea10-0b87-489d-bc67-fb9bd6837b89 None admin] RBAC: Authorizing `identity:get_role(role_id=c4bab4a02e5c4f5f97a024e844a0c072)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:06:08.307344 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a4ea10-0b87-489d-bc67-fb9bd6837b89 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:06:08.308731 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 569/1126] 10.222.0.26 () {56 vars in 1177 bytes} [Thu Jul 14 09:06:08 2022] GET /identity/v3/roles/c4bab4a02e5c4f5f97a024e844a0c072 => generated 223 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:06:08.315240 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6b6704-a69c-400e-b3a6-5cfc29ec7ec1 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:06:08.423006 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6b6704-a69c-400e-b3a6-5cfc29ec7ec1 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:06:08.461035 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee08e63c-0b53-4d6b-88e7-c22db9db12d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:06:08.464105 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee08e63c-0b53-4d6b-88e7-c22db9db12d1 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:06:08.466554 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee08e63c-0b53-4d6b-88e7-c22db9db12d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:06:08.466843 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee08e63c-0b53-4d6b-88e7-c22db9db12d1 None admin] PATH_INFO: `/v3/users/e2e3d43660964d0aadaff82d24e41bcf` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:06:08.468214 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee08e63c-0b53-4d6b-88e7-c22db9db12d1 None admin] RBAC: Authorizing `identity:get_user(user_id=e2e3d43660964d0aadaff82d24e41bcf)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:06:08.469666 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee08e63c-0b53-4d6b-88e7-c22db9db12d1 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:06:08.471378 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 558/1127] 10.222.0.26 () {56 vars in 1177 bytes} [Thu Jul 14 09:06:08 2022] GET /identity/v3/users/e2e3d43660964d0aadaff82d24e41bcf => generated 286 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:06:08.477428 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a4ea10-0b87-489d-bc67-fb9bd6837b89 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:06:08.524389 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:06:08.582928 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a4ea10-0b87-489d-bc67-fb9bd6837b89 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:06:08.618436 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a1f05c-34da-4051-a8a3-4c8baffa42ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:06:08.619081 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a1f05c-34da-4051-a8a3-4c8baffa42ce None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:06:08.619394 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a1f05c-34da-4051-a8a3-4c8baffa42ce None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:06:08.619669 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a1f05c-34da-4051-a8a3-4c8baffa42ce None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:06:08.688954 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a1f05c-34da-4051-a8a3-4c8baffa42ce None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:06:08.690406 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a1f05c-34da-4051-a8a3-4c8baffa42ce None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:06:08.739709 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:06:08.819178 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 570/1128] 10.222.0.26 () {58 vars in 1282 bytes} [Thu Jul 14 09:06:08 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 09:06:08.825729 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee08e63c-0b53-4d6b-88e7-c22db9db12d1 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:06:08.893473 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:06:08.932694 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee08e63c-0b53-4d6b-88e7-c22db9db12d1 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:06:08.970433 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed003d7-98ab-41af-9b44-4bd0f9297d32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:06:08.971812 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed003d7-98ab-41af-9b44-4bd0f9297d32 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:06:08.972257 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed003d7-98ab-41af-9b44-4bd0f9297d32 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:06:08.972689 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed003d7-98ab-41af-9b44-4bd0f9297d32 None admin] PATH_INFO: `/v3/projects/d1b7367661964cf5a35f29aebdfa4650` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:06:08.974852 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed003d7-98ab-41af-9b44-4bd0f9297d32 None admin] RBAC: Authorizing `identity:get_project(project_id=d1b7367661964cf5a35f29aebdfa4650)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:06:08.977238 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed003d7-98ab-41af-9b44-4bd0f9297d32 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:06:08.980035 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 559/1129] 10.222.0.26 () {56 vars in 1186 bytes} [Thu Jul 14 09:06:08 2022] GET /identity/v3/projects/d1b7367661964cf5a35f29aebdfa4650 => generated 311 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:06:08.989290 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a1f05c-34da-4051-a8a3-4c8baffa42ce None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:06:09.094391 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a1f05c-34da-4051-a8a3-4c8baffa42ce None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:06:09.130655 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5a122e-c103-4714-8c33-918ad0f588af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:06:09.131364 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5a122e-c103-4714-8c33-918ad0f588af None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:06:09.131669 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5a122e-c103-4714-8c33-918ad0f588af None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:06:09.131971 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5a122e-c103-4714-8c33-918ad0f588af None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:06:09.132856 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5a122e-c103-4714-8c33-918ad0f588af None admin] RBAC: Adding query filter params (role.id=c4bab4a02e5c4f5f97a024e844a0c072, scope.project.id=d1b7367661964cf5a35f29aebdfa4650, user.id=e2e3d43660964d0aadaff82d24e41bcf) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:06:09.133174 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5a122e-c103-4714-8c33-918ad0f588af None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:06:09.134733 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5a122e-c103-4714-8c33-918ad0f588af None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:06:09.140886 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 571/1130] 10.222.0.26 () {56 vars in 1374 bytes} [Thu Jul 14 09:06:08 2022] GET /identity/v3/role_assignments?user.id=e2e3d43660964d0aadaff82d24e41bcf&scope.project.id=d1b7367661964cf5a35f29aebdfa4650&role.id=c4bab4a02e5c4f5f97a024e844a0c072 => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:06:11.152251 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4db3f92-321a-439a-846d-efe2c1d74e76 None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:06:11.153140 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4db3f92-321a-439a-846d-efe2c1d74e76 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:06:11.153140 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4db3f92-321a-439a-846d-efe2c1d74e76 None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:06:11.155417 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 560/1131] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:06:11 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:06:11.161710 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a487209a-ef83-4941-9aaa-85f5e6811b7e None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:06:11.162509 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a487209a-ef83-4941-9aaa-85f5e6811b7e None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:06:11.162662 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a487209a-ef83-4941-9aaa-85f5e6811b7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:06:11.183018 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:06:11.188802 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-a487209a-ef83-4941-9aaa-85f5e6811b7e None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:06:11.190007 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-a487209a-ef83-4941-9aaa-85f5e6811b7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:06:11.261672 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 572/1132] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:06:11 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 09:06:11.285666 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4db3f92-321a-439a-846d-efe2c1d74e76 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:06:11.405613 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4db3f92-321a-439a-846d-efe2c1d74e76 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:06:11.445012 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe5c379-c0fe-4d71-b4cc-9e73ffc38c2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:06:11.445796 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe5c379-c0fe-4d71-b4cc-9e73ffc38c2c None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:06:11.446082 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe5c379-c0fe-4d71-b4cc-9e73ffc38c2c None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:06:11.446358 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe5c379-c0fe-4d71-b4cc-9e73ffc38c2c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:06:11.447738 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe5c379-c0fe-4d71-b4cc-9e73ffc38c2c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:06:11.453276 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe5c379-c0fe-4d71-b4cc-9e73ffc38c2c None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:06:11.458115 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 561/1133] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jul 14 09:06:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:06:11.464492 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a487209a-ef83-4941-9aaa-85f5e6811b7e None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:06:11.577524 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a487209a-ef83-4941-9aaa-85f5e6811b7e None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:06:11.613959 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ef8b51-c6d5-4802-8d7e-7d3d004503ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:06:11.614614 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ef8b51-c6d5-4802-8d7e-7d3d004503ec None admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:06:11.614894 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ef8b51-c6d5-4802-8d7e-7d3d004503ec None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:06:11.615168 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ef8b51-c6d5-4802-8d7e-7d3d004503ec None admin] PATH_INFO: `/v3/projects` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:06:11.615909 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ef8b51-c6d5-4802-8d7e-7d3d004503ec None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:06:11.617340 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ef8b51-c6d5-4802-8d7e-7d3d004503ec None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:06:11.621501 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:06:11.645734 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 573/1134] 10.222.0.26 () {60 vars in 1141 bytes} [Thu Jul 14 09:06:11 2022] POST /identity/v3/projects => generated 311 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 09:06:13.668974 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc48b3f-5059-4ec0-b588-2720b26a531d None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:06:13.669791 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc48b3f-5059-4ec0-b588-2720b26a531d None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:06:13.669964 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc48b3f-5059-4ec0-b588-2720b26a531d None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:06:13.672222 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 562/1135] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:06:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:06:13.678431 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dcb388-80d3-4a15-b0bc-22593860dc82 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:06:13.678862 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dcb388-80d3-4a15-b0bc-22593860dc82 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:06:13.678972 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dcb388-80d3-4a15-b0bc-22593860dc82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:06:13.705097 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-58dcb388-80d3-4a15-b0bc-22593860dc82 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:06:13.706271 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-58dcb388-80d3-4a15-b0bc-22593860dc82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:06:13.782686 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 574/1136] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:06:13 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 09:06:13.807835 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc48b3f-5059-4ec0-b588-2720b26a531d None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:06:13.811050 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:06:13.917643 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc48b3f-5059-4ec0-b588-2720b26a531d None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:06:13.955162 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0c6e0d-2016-40b7-b693-03d57ce8ee9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:06:13.955983 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0c6e0d-2016-40b7-b693-03d57ce8ee9f None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:06:13.956267 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0c6e0d-2016-40b7-b693-03d57ce8ee9f None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:06:13.956548 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0c6e0d-2016-40b7-b693-03d57ce8ee9f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:06:13.957856 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0c6e0d-2016-40b7-b693-03d57ce8ee9f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:06:13.959340 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0c6e0d-2016-40b7-b693-03d57ce8ee9f None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:06:13.960968 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 563/1137] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jul 14 09:06:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:06:13.967947 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dcb388-80d3-4a15-b0bc-22593860dc82 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:06:14.080852 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dcb388-80d3-4a15-b0bc-22593860dc82 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:06:14.098128 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:06:14.117476 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24a3262-a02f-4c74-a149-0ad8c065658d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:06:14.118143 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24a3262-a02f-4c74-a149-0ad8c065658d None admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:06:14.118424 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24a3262-a02f-4c74-a149-0ad8c065658d None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:06:14.118693 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24a3262-a02f-4c74-a149-0ad8c065658d None admin] PATH_INFO: `/v3/users` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:06:14.119433 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24a3262-a02f-4c74-a149-0ad8c065658d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:06:14.120864 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24a3262-a02f-4c74-a149-0ad8c065658d None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:06:14.142926 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 575/1138] 10.222.0.26 () {60 vars in 1132 bytes} [Thu Jul 14 09:06:13 2022] POST /identity/v3/users => generated 286 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 09:06:16.153587 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08831edf-8bf6-4842-b0d5-121b84594c57 None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:06:16.154149 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08831edf-8bf6-4842-b0d5-121b84594c57 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:06:16.154149 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08831edf-8bf6-4842-b0d5-121b84594c57 None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:06:16.155559 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 564/1139] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:06:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:06:16.161767 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7ba63e-bc7f-4e0e-86b8-b076218bf52e None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:06:16.162215 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7ba63e-bc7f-4e0e-86b8-b076218bf52e None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:06:16.162358 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7ba63e-bc7f-4e0e-86b8-b076218bf52e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:06:16.185776 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-bf7ba63e-bc7f-4e0e-86b8-b076218bf52e None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:06:16.186972 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-bf7ba63e-bc7f-4e0e-86b8-b076218bf52e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:06:16.257461 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 576/1140] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:06:16 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 09:06:16.285000 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08831edf-8bf6-4842-b0d5-121b84594c57 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:06:16.362512 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:06:16.399151 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08831edf-8bf6-4842-b0d5-121b84594c57 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:06:16.434256 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338b92dd-90c2-437d-b612-ca76d890ee45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:06:16.434975 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338b92dd-90c2-437d-b612-ca76d890ee45 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:06:16.435272 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338b92dd-90c2-437d-b612-ca76d890ee45 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:06:16.435548 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338b92dd-90c2-437d-b612-ca76d890ee45 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:06:16.445759 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338b92dd-90c2-437d-b612-ca76d890ee45 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 09:06:16.453426 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-338b92dd-90c2-437d-b612-ca76d890ee45 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 09:06:16.453426 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:06:16.453426 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:06:16.453426 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:06:16.453426 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:06:16.453426 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:06:16.453426 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:06:16.453426 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:06:16.453426 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:06:16.453426 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:06:16.453426 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:06:16.453426 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:06:16.453426 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 09:06:16.453426 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 09:06:16.453426 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 09:06:16.453426 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 09:06:16.454582 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 09:06:16.454582 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 09:06:16.454582 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 09:06:16.454582 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 09:06:16.454582 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:06:16.454582 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:06:16.454582 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:06:16.454582 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:06:16.454582 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:06:16.454582 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:06:16.454582 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:06:16.454582 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 14 09:06:16.454582 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:06:16.454582 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:06:16.454582 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:06:16.454582 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:06:16.455594 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:06:16.455594 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:06:16.455594 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:06:16.455594 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:06:16.455594 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 09:06:16.455594 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 09:06:16.455594 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 09:06:16.455594 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 09:06:16.455594 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 09:06:16.455594 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 09:06:16.455594 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 09:06:16.455594 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 14 09:06:16.455594 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 565/1141] 10.222.0.26 () {56 vars in 1096 bytes} [Thu Jul 14 09:06:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:06:16.462990 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7ba63e-bc7f-4e0e-86b8-b076218bf52e None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:06:16.574569 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7ba63e-bc7f-4e0e-86b8-b076218bf52e None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:06:16.576964 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:06:16.609203 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5c6c73-9c9d-4f1f-83ae-da614dcada91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:06:16.609860 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5c6c73-9c9d-4f1f-83ae-da614dcada91 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:06:16.610145 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5c6c73-9c9d-4f1f-83ae-da614dcada91 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:06:16.610423 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5c6c73-9c9d-4f1f-83ae-da614dcada91 None admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:06:16.611177 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5c6c73-9c9d-4f1f-83ae-da614dcada91 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:06:16.611521 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5c6c73-9c9d-4f1f-83ae-da614dcada91 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:06:16.612901 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5c6c73-9c9d-4f1f-83ae-da614dcada91 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:06:16.620936 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 577/1142] 10.222.0.26 () {56 vars in 1099 bytes} [Thu Jul 14 09:06:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:06:16.627079 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338b92dd-90c2-437d-b612-ca76d890ee45 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:06:16.737074 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338b92dd-90c2-437d-b612-ca76d890ee45 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:06:16.766285 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:06:16.773300 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b856df3e-dc65-45ec-b0e3-e4a19340ffef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:06:16.774200 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b856df3e-dc65-45ec-b0e3-e4a19340ffef None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:06:16.774457 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b856df3e-dc65-45ec-b0e3-e4a19340ffef None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:06:16.774732 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b856df3e-dc65-45ec-b0e3-e4a19340ffef None admin] PATH_INFO: `/v3/users/7ae4bc44256349dfb24807de24225fc2` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:06:16.789045 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b856df3e-dc65-45ec-b0e3-e4a19340ffef None admin] RBAC: Authorizing `identity:get_user(user_id=7ae4bc44256349dfb24807de24225fc2)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:06:16.790501 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b856df3e-dc65-45ec-b0e3-e4a19340ffef None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:06:16.792226 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 566/1143] 10.222.0.26 () {56 vars in 1177 bytes} [Thu Jul 14 09:06:16 2022] GET /identity/v3/users/7ae4bc44256349dfb24807de24225fc2 => generated 286 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:06:16.798759 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5c6c73-9c9d-4f1f-83ae-da614dcada91 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:06:16.906842 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5c6c73-9c9d-4f1f-83ae-da614dcada91 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:06:16.944091 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddad6a8c-de69-4bf7-b0c1-776f689829ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:06:16.944729 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddad6a8c-de69-4bf7-b0c1-776f689829ad None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:06:16.945007 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddad6a8c-de69-4bf7-b0c1-776f689829ad None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:06:16.945268 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddad6a8c-de69-4bf7-b0c1-776f689829ad None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:06:16.980637 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:06:17.017507 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddad6a8c-de69-4bf7-b0c1-776f689829ad None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:06:17.018879 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddad6a8c-de69-4bf7-b0c1-776f689829ad None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:06:17.127764 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 578/1144] 10.222.0.26 () {58 vars in 1282 bytes} [Thu Jul 14 09:06:16 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 09:06:17.133841 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b856df3e-dc65-45ec-b0e3-e4a19340ffef None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:06:17.237744 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b856df3e-dc65-45ec-b0e3-e4a19340ffef None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:06:17.272579 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ebd0e2-b440-4eb6-a7d1-428d32a91a2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:06:17.273369 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ebd0e2-b440-4eb6-a7d1-428d32a91a2e None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:06:17.273642 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ebd0e2-b440-4eb6-a7d1-428d32a91a2e None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:06:17.273912 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ebd0e2-b440-4eb6-a7d1-428d32a91a2e None admin] PATH_INFO: `/v3/projects/569ccdb3d4e94e98bf761a95fc49fa37` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:06:17.275108 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ebd0e2-b440-4eb6-a7d1-428d32a91a2e None admin] RBAC: Authorizing `identity:get_project(project_id=569ccdb3d4e94e98bf761a95fc49fa37)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:06:17.276566 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ebd0e2-b440-4eb6-a7d1-428d32a91a2e None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:06:17.278230 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 567/1145] 10.222.0.26 () {56 vars in 1186 bytes} [Thu Jul 14 09:06:17 2022] GET /identity/v3/projects/569ccdb3d4e94e98bf761a95fc49fa37 => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:06:17.284437 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddad6a8c-de69-4bf7-b0c1-776f689829ad None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:06:17.355759 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:06:17.390917 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddad6a8c-de69-4bf7-b0c1-776f689829ad None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:06:17.429510 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02db0aff-a08e-4dc2-a7a8-a680da45f61f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:06:17.430141 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02db0aff-a08e-4dc2-a7a8-a680da45f61f None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:06:17.430419 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02db0aff-a08e-4dc2-a7a8-a680da45f61f None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:06:17.430673 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02db0aff-a08e-4dc2-a7a8-a680da45f61f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:06:17.431534 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02db0aff-a08e-4dc2-a7a8-a680da45f61f None admin] RBAC: Adding query filter params (role.id=7b549d58d1724395b343609d895c80d0, scope.project.id=569ccdb3d4e94e98bf761a95fc49fa37, user.id=7ae4bc44256349dfb24807de24225fc2) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:06:17.431837 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02db0aff-a08e-4dc2-a7a8-a680da45f61f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:06:17.433254 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02db0aff-a08e-4dc2-a7a8-a680da45f61f None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:06:17.438637 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 579/1146] 10.222.0.26 () {56 vars in 1374 bytes} [Thu Jul 14 09:06:17 2022] GET /identity/v3/role_assignments?user.id=7ae4bc44256349dfb24807de24225fc2&scope.project.id=569ccdb3d4e94e98bf761a95fc49fa37&role.id=7b549d58d1724395b343609d895c80d0 => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:06:19.397767 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db714c95-abab-4ee3-ac0a-52200ab7aabf None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:06:19.398363 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db714c95-abab-4ee3-ac0a-52200ab7aabf None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:06:19.398363 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db714c95-abab-4ee3-ac0a-52200ab7aabf None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:06:19.399749 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 568/1147] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:06:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:06:19.406074 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7768341-7a85-42bb-8f98-4a3ab306144f None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:06:19.406789 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7768341-7a85-42bb-8f98-4a3ab306144f None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:06:19.406976 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7768341-7a85-42bb-8f98-4a3ab306144f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:06:19.430216 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-c7768341-7a85-42bb-8f98-4a3ab306144f None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:06:19.431443 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-c7768341-7a85-42bb-8f98-4a3ab306144f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:06:19.509582 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 580/1148] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:06:19 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 09:06:19.532515 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db714c95-abab-4ee3-ac0a-52200ab7aabf None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:06:19.595314 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:06:19.641505 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db714c95-abab-4ee3-ac0a-52200ab7aabf None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:06:19.676647 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb072d2-ef6b-46c6-a839-13420a934f66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:06:19.677366 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb072d2-ef6b-46c6-a839-13420a934f66 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:06:19.677647 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb072d2-ef6b-46c6-a839-13420a934f66 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:06:19.677925 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb072d2-ef6b-46c6-a839-13420a934f66 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:06:19.689287 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb072d2-ef6b-46c6-a839-13420a934f66 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 09:06:19.696889 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-ffb072d2-ef6b-46c6-a839-13420a934f66 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 09:06:19.696889 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:06:19.696889 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:06:19.696889 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:06:19.696889 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:06:19.696889 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:06:19.696889 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:06:19.696889 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:06:19.696889 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:06:19.696889 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:06:19.696889 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:06:19.696889 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:06:19.696889 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 09:06:19.696889 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 09:06:19.696889 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 09:06:19.696889 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 09:06:19.697966 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 09:06:19.697966 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 09:06:19.697966 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 09:06:19.697966 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 09:06:19.697966 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:06:19.697966 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:06:19.697966 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:06:19.697966 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:06:19.697966 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:06:19.697966 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:06:19.697966 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:06:19.697966 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 14 09:06:19.697966 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:06:19.697966 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:06:19.697966 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:06:19.697966 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:06:19.698992 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:06:19.698992 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:06:19.698992 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:06:19.698992 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:06:19.698992 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 09:06:19.698992 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 09:06:19.698992 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 09:06:19.698992 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 09:06:19.698992 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 09:06:19.698992 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 09:06:19.698992 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 09:06:19.698992 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 14 09:06:19.703111 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 569/1149] 10.222.0.26 () {56 vars in 1096 bytes} [Thu Jul 14 09:06:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:06:19.704226 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7768341-7a85-42bb-8f98-4a3ab306144f None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:06:19.763988 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:06:19.812057 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7768341-7a85-42bb-8f98-4a3ab306144f None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:06:19.850060 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6578db37-2dc2-4f3a-8d31-1b80ff83db38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:06:19.850712 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6578db37-2dc2-4f3a-8d31-1b80ff83db38 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:06:19.850994 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6578db37-2dc2-4f3a-8d31-1b80ff83db38 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:06:19.851300 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6578db37-2dc2-4f3a-8d31-1b80ff83db38 None admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:06:19.852050 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6578db37-2dc2-4f3a-8d31-1b80ff83db38 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:06:19.852358 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6578db37-2dc2-4f3a-8d31-1b80ff83db38 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:06:19.853732 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6578db37-2dc2-4f3a-8d31-1b80ff83db38 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:06:19.862127 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 581/1150] 10.222.0.26 () {56 vars in 1099 bytes} [Thu Jul 14 09:06:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:06:19.868178 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb072d2-ef6b-46c6-a839-13420a934f66 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:06:19.987462 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb072d2-ef6b-46c6-a839-13420a934f66 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:06:20.003434 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:06:20.022586 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792ab472-ab1f-4a04-bd07-81442a73e80b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:06:20.023469 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792ab472-ab1f-4a04-bd07-81442a73e80b None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:06:20.023749 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792ab472-ab1f-4a04-bd07-81442a73e80b None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:06:20.024021 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792ab472-ab1f-4a04-bd07-81442a73e80b None admin] PATH_INFO: `/v3/users/7ae4bc44256349dfb24807de24225fc2` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:06:20.025239 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792ab472-ab1f-4a04-bd07-81442a73e80b None admin] RBAC: Authorizing `identity:get_user(user_id=7ae4bc44256349dfb24807de24225fc2)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:06:20.026677 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792ab472-ab1f-4a04-bd07-81442a73e80b None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:06:20.028353 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 570/1151] 10.222.0.26 () {56 vars in 1177 bytes} [Thu Jul 14 09:06:19 2022] GET /identity/v3/users/7ae4bc44256349dfb24807de24225fc2 => generated 286 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:06:20.036770 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6578db37-2dc2-4f3a-8d31-1b80ff83db38 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:06:20.146104 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6578db37-2dc2-4f3a-8d31-1b80ff83db38 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:06:20.169144 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:06:20.181985 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4f3147-f304-4cd5-a635-59c8f12bd355 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:06:20.182841 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4f3147-f304-4cd5-a635-59c8f12bd355 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:06:20.183122 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4f3147-f304-4cd5-a635-59c8f12bd355 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:06:20.183431 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4f3147-f304-4cd5-a635-59c8f12bd355 None admin] PATH_INFO: `/v3/projects/569ccdb3d4e94e98bf761a95fc49fa37` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:06:20.184962 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4f3147-f304-4cd5-a635-59c8f12bd355 None admin] RBAC: Authorizing `identity:get_project(project_id=569ccdb3d4e94e98bf761a95fc49fa37)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:06:20.186430 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4f3147-f304-4cd5-a635-59c8f12bd355 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:06:20.188154 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 582/1152] 10.222.0.26 () {56 vars in 1186 bytes} [Thu Jul 14 09:06:20 2022] GET /identity/v3/projects/569ccdb3d4e94e98bf761a95fc49fa37 => generated 311 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:06:20.194381 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792ab472-ab1f-4a04-bd07-81442a73e80b None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:06:20.301767 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792ab472-ab1f-4a04-bd07-81442a73e80b None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:06:20.339939 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eed3e70-a49c-490d-92cd-d2461d182821 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:06:20.340841 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eed3e70-a49c-490d-92cd-d2461d182821 None admin] REQUEST_METHOD: `PUT` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:06:20.341125 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eed3e70-a49c-490d-92cd-d2461d182821 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:06:20.341403 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eed3e70-a49c-490d-92cd-d2461d182821 None admin] PATH_INFO: `/v3/projects/569ccdb3d4e94e98bf761a95fc49fa37/users/7ae4bc44256349dfb24807de24225fc2/roles/7b549d58d1724395b343609d895c80d0` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:06:20.343658 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eed3e70-a49c-490d-92cd-d2461d182821 None admin] RBAC: Authorizing `identity:create_grant(project_id=569ccdb3d4e94e98bf761a95fc49fa37, user_id=7ae4bc44256349dfb24807de24225fc2, role_id=7b549d58d1724395b343609d895c80d0)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:06:20.345074 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eed3e70-a49c-490d-92cd-d2461d182821 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:06:20.362992 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 571/1153] 10.222.0.26 () {58 vars in 1439 bytes} [Thu Jul 14 09:06:20 2022] PUT /identity/v3/projects/569ccdb3d4e94e98bf761a95fc49fa37/users/7ae4bc44256349dfb24807de24225fc2/roles/7b549d58d1724395b343609d895c80d0 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:06:22.352613 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a6ea4d-e93f-457d-9a5d-886f0a9f5814 None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:06:22.353171 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a6ea4d-e93f-457d-9a5d-886f0a9f5814 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:06:22.353171 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a6ea4d-e93f-457d-9a5d-886f0a9f5814 None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:06:22.354563 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 583/1154] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:06:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:06:22.361036 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4969f6ac-2246-4f56-9bc6-07fd4a031704 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:06:22.361751 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4969f6ac-2246-4f56-9bc6-07fd4a031704 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:06:22.361888 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4969f6ac-2246-4f56-9bc6-07fd4a031704 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:06:22.390372 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-4969f6ac-2246-4f56-9bc6-07fd4a031704 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:06:22.391979 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-4969f6ac-2246-4f56-9bc6-07fd4a031704 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:06:22.438809 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:06:22.472568 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 572/1155] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:06:22 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 09:06:22.496140 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a6ea4d-e93f-457d-9a5d-886f0a9f5814 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:06:22.612748 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a6ea4d-e93f-457d-9a5d-886f0a9f5814 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:06:22.647503 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114b9b74-7dd7-43a3-a1f8-d3573ad73e0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:06:22.648289 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114b9b74-7dd7-43a3-a1f8-d3573ad73e0e None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:06:22.648572 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114b9b74-7dd7-43a3-a1f8-d3573ad73e0e None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:06:22.648842 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114b9b74-7dd7-43a3-a1f8-d3573ad73e0e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:06:22.659496 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114b9b74-7dd7-43a3-a1f8-d3573ad73e0e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 09:06:22.667145 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-114b9b74-7dd7-43a3-a1f8-d3573ad73e0e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 09:06:22.667145 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:06:22.667145 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:06:22.667145 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:06:22.667145 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:06:22.667145 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:06:22.667145 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:06:22.667145 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:06:22.667145 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:06:22.667145 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:06:22.667145 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:06:22.667145 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:06:22.667145 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 09:06:22.667145 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 09:06:22.667145 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 09:06:22.667145 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 09:06:22.668283 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 09:06:22.668283 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 09:06:22.668283 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 09:06:22.668283 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 09:06:22.668283 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:06:22.668283 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:06:22.668283 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:06:22.668283 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:06:22.668283 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:06:22.668283 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:06:22.668283 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:06:22.668283 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 14 09:06:22.668283 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:06:22.668283 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:06:22.668283 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:06:22.668283 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:06:22.669686 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:06:22.669686 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:06:22.669686 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:06:22.669686 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:06:22.669686 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 09:06:22.669686 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 09:06:22.669686 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 09:06:22.669686 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 09:06:22.669686 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 09:06:22.669686 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 09:06:22.669686 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 09:06:22.669686 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 14 09:06:22.669686 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 584/1156] 10.222.0.26 () {56 vars in 1096 bytes} [Thu Jul 14 09:06:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:06:22.677298 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4969f6ac-2246-4f56-9bc6-07fd4a031704 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:06:22.788059 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4969f6ac-2246-4f56-9bc6-07fd4a031704 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:06:22.824826 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cf863d-7072-4368-935f-e5da79d94c0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:06:22.825491 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cf863d-7072-4368-935f-e5da79d94c0c None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:06:22.825778 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cf863d-7072-4368-935f-e5da79d94c0c None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:06:22.826049 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cf863d-7072-4368-935f-e5da79d94c0c None admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:06:22.826811 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cf863d-7072-4368-935f-e5da79d94c0c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:06:22.827116 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cf863d-7072-4368-935f-e5da79d94c0c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:06:22.828537 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cf863d-7072-4368-935f-e5da79d94c0c None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:06:22.837941 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 573/1157] 10.222.0.26 () {56 vars in 1099 bytes} [Thu Jul 14 09:06:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:06:22.844847 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114b9b74-7dd7-43a3-a1f8-d3573ad73e0e None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:06:22.902341 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:06:22.958548 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114b9b74-7dd7-43a3-a1f8-d3573ad73e0e None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:06:22.994061 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb393871-9c61-476d-9845-cd6ea4265c8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:06:22.994940 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb393871-9c61-476d-9845-cd6ea4265c8e None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:06:22.995236 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb393871-9c61-476d-9845-cd6ea4265c8e None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:06:22.995534 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb393871-9c61-476d-9845-cd6ea4265c8e None admin] PATH_INFO: `/v3/users/7ae4bc44256349dfb24807de24225fc2` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:06:22.996841 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb393871-9c61-476d-9845-cd6ea4265c8e None admin] RBAC: Authorizing `identity:get_user(user_id=7ae4bc44256349dfb24807de24225fc2)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:06:22.998290 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb393871-9c61-476d-9845-cd6ea4265c8e None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:06:22.999992 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 585/1158] 10.222.0.26 () {56 vars in 1177 bytes} [Thu Jul 14 09:06:22 2022] GET /identity/v3/users/7ae4bc44256349dfb24807de24225fc2 => generated 286 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:06:23.006352 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cf863d-7072-4368-935f-e5da79d94c0c None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:06:23.055328 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:06:23.124647 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cf863d-7072-4368-935f-e5da79d94c0c None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:06:23.161407 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d116f0d3-3889-40bf-a382-956d386c2ab2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:06:23.161988 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d116f0d3-3889-40bf-a382-956d386c2ab2 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:06:23.162258 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d116f0d3-3889-40bf-a382-956d386c2ab2 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:06:23.162533 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d116f0d3-3889-40bf-a382-956d386c2ab2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:06:23.239210 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d116f0d3-3889-40bf-a382-956d386c2ab2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:06:23.240676 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d116f0d3-3889-40bf-a382-956d386c2ab2 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:06:23.281558 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:06:23.350877 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 574/1159] 10.222.0.26 () {58 vars in 1282 bytes} [Thu Jul 14 09:06:23 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 09:06:23.357600 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb393871-9c61-476d-9845-cd6ea4265c8e None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:06:23.465992 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb393871-9c61-476d-9845-cd6ea4265c8e None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:06:23.482999 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:06:23.503840 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230fb362-9fcf-4ef6-8d27-c7f84fed2a4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:06:23.504684 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230fb362-9fcf-4ef6-8d27-c7f84fed2a4d None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:06:23.504968 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230fb362-9fcf-4ef6-8d27-c7f84fed2a4d None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:06:23.505240 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230fb362-9fcf-4ef6-8d27-c7f84fed2a4d None admin] PATH_INFO: `/v3/projects/569ccdb3d4e94e98bf761a95fc49fa37` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:06:23.506475 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230fb362-9fcf-4ef6-8d27-c7f84fed2a4d None admin] RBAC: Authorizing `identity:get_project(project_id=569ccdb3d4e94e98bf761a95fc49fa37)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:06:23.507947 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230fb362-9fcf-4ef6-8d27-c7f84fed2a4d None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:06:23.509663 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 586/1160] 10.222.0.26 () {56 vars in 1186 bytes} [Thu Jul 14 09:06:23 2022] GET /identity/v3/projects/569ccdb3d4e94e98bf761a95fc49fa37 => generated 311 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:06:23.516163 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d116f0d3-3889-40bf-a382-956d386c2ab2 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:06:23.625904 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d116f0d3-3889-40bf-a382-956d386c2ab2 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:06:23.660965 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a075753-0a61-49b9-a23a-d64dda1fce7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:06:23.661610 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a075753-0a61-49b9-a23a-d64dda1fce7e None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:06:23.661890 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a075753-0a61-49b9-a23a-d64dda1fce7e None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:06:23.662166 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a075753-0a61-49b9-a23a-d64dda1fce7e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:06:23.663001 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a075753-0a61-49b9-a23a-d64dda1fce7e None admin] RBAC: Adding query filter params (role.id=7b549d58d1724395b343609d895c80d0, scope.project.id=569ccdb3d4e94e98bf761a95fc49fa37, user.id=7ae4bc44256349dfb24807de24225fc2) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:06:23.663344 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a075753-0a61-49b9-a23a-d64dda1fce7e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:06:23.664772 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a075753-0a61-49b9-a23a-d64dda1fce7e None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:06:23.667688 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:06:23.671131 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 575/1161] 10.222.0.26 () {56 vars in 1374 bytes} [Thu Jul 14 09:06:23 2022] GET /identity/v3/role_assignments?user.id=7ae4bc44256349dfb24807de24225fc2&scope.project.id=569ccdb3d4e94e98bf761a95fc49fa37&role.id=7b549d58d1724395b343609d895c80d0 => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:06:25.680576 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc8e8d3-0eed-46a8-98dd-87767c72e8dd None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:06:25.681457 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc8e8d3-0eed-46a8-98dd-87767c72e8dd None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:06:25.681457 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc8e8d3-0eed-46a8-98dd-87767c72e8dd None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:06:25.683759 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 587/1162] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:06:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:06:25.692497 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6057614b-c1ec-4bb6-9e84-cc330a308255 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:06:25.693354 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6057614b-c1ec-4bb6-9e84-cc330a308255 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:06:25.693455 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6057614b-c1ec-4bb6-9e84-cc330a308255 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:06:25.712993 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-6057614b-c1ec-4bb6-9e84-cc330a308255 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:06:25.714179 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-6057614b-c1ec-4bb6-9e84-cc330a308255 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:06:25.788773 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 576/1163] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:06:25 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 09:06:25.811814 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc8e8d3-0eed-46a8-98dd-87767c72e8dd None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:06:25.934925 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc8e8d3-0eed-46a8-98dd-87767c72e8dd None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:06:25.970050 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c70821-9930-4ff2-8646-9662ce8a832c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:06:25.970689 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c70821-9930-4ff2-8646-9662ce8a832c None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:06:25.970970 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c70821-9930-4ff2-8646-9662ce8a832c None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:06:25.971321 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c70821-9930-4ff2-8646-9662ce8a832c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:06:26.028428 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:06:26.055778 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c70821-9930-4ff2-8646-9662ce8a832c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:06:26.057265 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c70821-9930-4ff2-8646-9662ce8a832c None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:06:26.163922 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 588/1164] 10.222.0.26 () {58 vars in 1282 bytes} [Thu Jul 14 09:06:25 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 09:06:26.169941 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6057614b-c1ec-4bb6-9e84-cc330a308255 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:06:26.274669 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6057614b-c1ec-4bb6-9e84-cc330a308255 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:06:26.303246 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:06:26.310867 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da86bba-32a2-49bc-bc23-d27380833243 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:06:26.311679 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da86bba-32a2-49bc-bc23-d27380833243 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:06:26.311957 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da86bba-32a2-49bc-bc23-d27380833243 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:06:26.312224 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da86bba-32a2-49bc-bc23-d27380833243 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:06:26.319992 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da86bba-32a2-49bc-bc23-d27380833243 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:06:26.321419 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da86bba-32a2-49bc-bc23-d27380833243 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:06:26.336190 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-5da86bba-32a2-49bc-bc23-d27380833243 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 14 09:06:26.336190 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:06:26.336190 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jul 14 09:06:26.336190 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jul 14 09:06:26.336190 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 09:06:26.336190 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 09:06:26.336190 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 09:06:26.336190 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 09:06:26.336190 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jul 14 09:06:26.336190 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 14 09:06:26.336190 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 09:06:26.336190 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 14 09:06:26.336190 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:06:26.336190 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:06:26.336190 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:06:26.336190 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:06:26.336190 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:06:26.337380 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:06:26.337380 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:06:26.337380 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:06:26.337380 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:06:26.337380 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:06:26.337380 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:06:26.337380 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 09:06:26.337380 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 09:06:26.337380 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 09:06:26.337380 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 09:06:26.337380 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:06:26.337380 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:06:26.337380 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:06:26.337380 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:06:26.337380 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:06:26.337380 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:06:26.338481 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:06:26.338481 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 14 09:06:26.338481 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:06:26.338481 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:06:26.338481 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:06:26.338481 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:06:26.338481 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:06:26.338481 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:06:26.338481 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:06:26.338481 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:06:26.338481 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jul 14 09:06:26.338481 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 09:06:26.338481 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 14 09:06:26.338481 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 14 09:06:26.338481 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 577/1165] 10.222.0.26 () {56 vars in 1117 bytes} [Thu Jul 14 09:06:26 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:06:26.343414 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c70821-9930-4ff2-8646-9662ce8a832c None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:06:26.430750 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:06:26.450800 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c70821-9930-4ff2-8646-9662ce8a832c None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:06:26.490295 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9e34f1-2fa3-4594-939b-238fbb16462a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:06:26.490958 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9e34f1-2fa3-4594-939b-238fbb16462a None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:06:26.491363 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9e34f1-2fa3-4594-939b-238fbb16462a None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:06:26.491792 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9e34f1-2fa3-4594-939b-238fbb16462a None admin] PATH_INFO: `/v3/domains` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:06:26.492832 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9e34f1-2fa3-4594-939b-238fbb16462a None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:06:26.493146 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9e34f1-2fa3-4594-939b-238fbb16462a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:06:26.494531 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9e34f1-2fa3-4594-939b-238fbb16462a None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:06:26.501961 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 589/1166] 10.222.0.26 () {56 vars in 1115 bytes} [Thu Jul 14 09:06:26 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:06:28.508870 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68585eb0-02f6-4f42-a5cd-eca985637cbd None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:06:28.509424 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68585eb0-02f6-4f42-a5cd-eca985637cbd None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:06:28.509424 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68585eb0-02f6-4f42-a5cd-eca985637cbd None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:06:28.511603 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 578/1167] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:06:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:06:28.520395 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96587c46-b82a-4218-b7c6-56050c19270b None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:06:28.520815 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96587c46-b82a-4218-b7c6-56050c19270b None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:06:28.521008 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96587c46-b82a-4218-b7c6-56050c19270b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:06:28.541250 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-96587c46-b82a-4218-b7c6-56050c19270b None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:06:28.542445 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-96587c46-b82a-4218-b7c6-56050c19270b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:06:28.615111 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 590/1168] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:06:28 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 09:06:28.638011 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68585eb0-02f6-4f42-a5cd-eca985637cbd None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:06:28.755212 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68585eb0-02f6-4f42-a5cd-eca985637cbd None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:06:28.789490 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b94d730-95e8-4dfc-8375-d0f45bb5d705 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:06:28.790060 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b94d730-95e8-4dfc-8375-d0f45bb5d705 None admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:06:28.790343 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b94d730-95e8-4dfc-8375-d0f45bb5d705 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:06:28.790613 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b94d730-95e8-4dfc-8375-d0f45bb5d705 None admin] PATH_INFO: `/v3/domains` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:06:28.791394 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b94d730-95e8-4dfc-8375-d0f45bb5d705 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:06:28.792787 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b94d730-95e8-4dfc-8375-d0f45bb5d705 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:06:28.816099 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 579/1169] 10.222.0.26 () {60 vars in 1138 bytes} [Thu Jul 14 09:06:28 2022] POST /identity/v3/domains => generated 267 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 09:06:30.840874 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17501ecb-1ac6-4da0-a554-3b62c8157700 None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:06:30.841756 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17501ecb-1ac6-4da0-a554-3b62c8157700 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:06:30.841756 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17501ecb-1ac6-4da0-a554-3b62c8157700 None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:06:30.844009 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 591/1170] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:06:30 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:06:30.850186 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29594edf-d02f-4655-8749-50f76596fe7a None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:06:30.850627 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29594edf-d02f-4655-8749-50f76596fe7a None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:06:30.850758 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29594edf-d02f-4655-8749-50f76596fe7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:06:30.876413 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-29594edf-d02f-4655-8749-50f76596fe7a None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:06:30.877614 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-29594edf-d02f-4655-8749-50f76596fe7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:06:30.926166 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:06:30.951852 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 580/1171] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:06:30 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 09:06:30.978274 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17501ecb-1ac6-4da0-a554-3b62c8157700 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:06:31.065080 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:06:31.090638 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17501ecb-1ac6-4da0-a554-3b62c8157700 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:06:31.126011 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163fc742-3f2d-4dd5-a266-899762326eba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:06:31.126813 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163fc742-3f2d-4dd5-a266-899762326eba None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:06:31.127096 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163fc742-3f2d-4dd5-a266-899762326eba None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:06:31.127410 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163fc742-3f2d-4dd5-a266-899762326eba None admin] PATH_INFO: `/v3/domains/30d25d80719d406db7492e64eaa1ca08` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:06:31.138451 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163fc742-3f2d-4dd5-a266-899762326eba None admin] RBAC: Authorizing `identity:get_domain(domain_id=30d25d80719d406db7492e64eaa1ca08)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:06:31.139953 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163fc742-3f2d-4dd5-a266-899762326eba None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:06:31.141603 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 592/1172] 10.222.0.26 () {56 vars in 1183 bytes} [Thu Jul 14 09:06:30 2022] GET /identity/v3/domains/30d25d80719d406db7492e64eaa1ca08 => generated 267 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:06:31.147686 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29594edf-d02f-4655-8749-50f76596fe7a None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:06:31.249628 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29594edf-d02f-4655-8749-50f76596fe7a None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:06:31.285366 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103dd1e4-c0eb-4bdc-919c-f3cbb1e42baf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:06:31.286016 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103dd1e4-c0eb-4bdc-919c-f3cbb1e42baf None admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:06:31.286298 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103dd1e4-c0eb-4bdc-919c-f3cbb1e42baf None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:06:31.286571 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103dd1e4-c0eb-4bdc-919c-f3cbb1e42baf None admin] PATH_INFO: `/v3/projects` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:06:31.287319 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103dd1e4-c0eb-4bdc-919c-f3cbb1e42baf None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:06:31.288826 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103dd1e4-c0eb-4bdc-919c-f3cbb1e42baf None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:06:31.322591 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 581/1173] 10.222.0.26 () {60 vars in 1141 bytes} [Thu Jul 14 09:06:31 2022] POST /identity/v3/projects => generated 361 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 09:06:33.355548 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da57549-082e-4428-8451-1a9e38bc4dd4 None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:06:33.356438 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da57549-082e-4428-8451-1a9e38bc4dd4 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:06:33.356438 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da57549-082e-4428-8451-1a9e38bc4dd4 None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:06:33.358684 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 593/1174] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:06:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:06:33.365206 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b7550f-70a3-445f-a7ab-f1b62b3fbad6 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:06:33.365668 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b7550f-70a3-445f-a7ab-f1b62b3fbad6 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:06:33.365763 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b7550f-70a3-445f-a7ab-f1b62b3fbad6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:06:33.393163 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-40b7550f-70a3-445f-a7ab-f1b62b3fbad6 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:06:33.394347 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-40b7550f-70a3-445f-a7ab-f1b62b3fbad6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:06:33.422017 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:06:33.467926 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 582/1175] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:06:33 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 09:06:33.493262 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da57549-082e-4428-8451-1a9e38bc4dd4 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:06:33.614656 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da57549-082e-4428-8451-1a9e38bc4dd4 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:06:33.649582 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11745bb0-c7ec-4379-80e7-b54f67684ab6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:06:33.650452 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11745bb0-c7ec-4379-80e7-b54f67684ab6 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:06:33.650742 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11745bb0-c7ec-4379-80e7-b54f67684ab6 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:06:33.651013 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11745bb0-c7ec-4379-80e7-b54f67684ab6 None admin] PATH_INFO: `/v3/domains/30d25d80719d406db7492e64eaa1ca08` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:06:33.652347 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11745bb0-c7ec-4379-80e7-b54f67684ab6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=30d25d80719d406db7492e64eaa1ca08)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:06:33.653802 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11745bb0-c7ec-4379-80e7-b54f67684ab6 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:06:33.655498 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 594/1176] 10.222.0.26 () {56 vars in 1183 bytes} [Thu Jul 14 09:06:33 2022] GET /identity/v3/domains/30d25d80719d406db7492e64eaa1ca08 => generated 267 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:06:33.661583 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b7550f-70a3-445f-a7ab-f1b62b3fbad6 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:06:33.767072 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b7550f-70a3-445f-a7ab-f1b62b3fbad6 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:06:33.802127 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6e5be1-24d6-47ce-8bca-fb017448509e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:06:33.802772 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6e5be1-24d6-47ce-8bca-fb017448509e None admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:06:33.803058 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6e5be1-24d6-47ce-8bca-fb017448509e None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:06:33.803331 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6e5be1-24d6-47ce-8bca-fb017448509e None admin] PATH_INFO: `/v3/users` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:06:33.804071 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6e5be1-24d6-47ce-8bca-fb017448509e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:06:33.805490 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6e5be1-24d6-47ce-8bca-fb017448509e None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:06:33.828154 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 583/1177] 10.222.0.26 () {60 vars in 1132 bytes} [Thu Jul 14 09:06:33 2022] POST /identity/v3/users => generated 311 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 09:06:36.035927 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e460bf-a823-4039-b8a3-ee003a5d9de6 None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:06:36.036482 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e460bf-a823-4039-b8a3-ee003a5d9de6 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:06:36.036482 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e460bf-a823-4039-b8a3-ee003a5d9de6 None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:06:36.038277 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 595/1178] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:06:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:06:36.043639 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63da91f-3576-417f-82f5-6dcb0d48486b None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:06:36.044047 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63da91f-3576-417f-82f5-6dcb0d48486b None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:06:36.044191 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63da91f-3576-417f-82f5-6dcb0d48486b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:06:36.064035 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-e63da91f-3576-417f-82f5-6dcb0d48486b None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:06:36.065346 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-e63da91f-3576-417f-82f5-6dcb0d48486b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:06:36.079542 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:06:36.140337 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 584/1179] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:06:36 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 09:06:36.164371 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e460bf-a823-4039-b8a3-ee003a5d9de6 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:06:36.175742 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:06:36.271636 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e460bf-a823-4039-b8a3-ee003a5d9de6 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:06:36.308286 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236109ae-bcac-4c51-8073-11a1e72d7f92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:06:36.309010 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236109ae-bcac-4c51-8073-11a1e72d7f92 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:06:36.309291 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236109ae-bcac-4c51-8073-11a1e72d7f92 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:06:36.309566 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236109ae-bcac-4c51-8073-11a1e72d7f92 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:06:36.320237 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236109ae-bcac-4c51-8073-11a1e72d7f92 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 09:06:36.327968 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-236109ae-bcac-4c51-8073-11a1e72d7f92 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 09:06:36.327968 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:06:36.327968 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:06:36.327968 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:06:36.327968 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:06:36.327968 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:06:36.327968 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:06:36.327968 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:06:36.327968 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:06:36.327968 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:06:36.327968 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:06:36.327968 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:06:36.327968 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 09:06:36.327968 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 09:06:36.327968 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 09:06:36.327968 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 09:06:36.329158 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 09:06:36.329158 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 09:06:36.329158 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 09:06:36.329158 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 09:06:36.329158 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:06:36.329158 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:06:36.329158 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:06:36.329158 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:06:36.329158 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:06:36.329158 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:06:36.329158 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:06:36.329158 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 14 09:06:36.329158 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:06:36.329158 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:06:36.329158 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:06:36.329158 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:06:36.330151 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:06:36.330151 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:06:36.330151 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:06:36.330151 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:06:36.330151 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 09:06:36.330151 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 09:06:36.330151 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 09:06:36.330151 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 09:06:36.330151 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 09:06:36.330151 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 09:06:36.330151 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 09:06:36.330151 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 14 09:06:36.330151 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 596/1180] 10.222.0.26 () {56 vars in 1096 bytes} [Thu Jul 14 09:06:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:06:36.336066 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63da91f-3576-417f-82f5-6dcb0d48486b None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:06:36.444459 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63da91f-3576-417f-82f5-6dcb0d48486b None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:06:36.480126 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fb94f4-c9cc-4cba-ba66-5f0e1a99fcb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:06:36.480783 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fb94f4-c9cc-4cba-ba66-5f0e1a99fcb0 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:06:36.481072 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fb94f4-c9cc-4cba-ba66-5f0e1a99fcb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:06:36.481344 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fb94f4-c9cc-4cba-ba66-5f0e1a99fcb0 None admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:06:36.482090 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fb94f4-c9cc-4cba-ba66-5f0e1a99fcb0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:06:36.482396 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fb94f4-c9cc-4cba-ba66-5f0e1a99fcb0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:06:36.483811 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fb94f4-c9cc-4cba-ba66-5f0e1a99fcb0 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:06:36.492354 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 585/1181] 10.222.0.26 () {56 vars in 1099 bytes} [Thu Jul 14 09:06:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:06:36.498851 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236109ae-bcac-4c51-8073-11a1e72d7f92 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:06:36.576422 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:06:36.612463 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236109ae-bcac-4c51-8073-11a1e72d7f92 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:06:36.650451 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1263a975-cc84-43dc-90ed-191d45727dcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:06:36.651617 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1263a975-cc84-43dc-90ed-191d45727dcd None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:06:36.651898 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1263a975-cc84-43dc-90ed-191d45727dcd None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:06:36.652176 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1263a975-cc84-43dc-90ed-191d45727dcd None admin] PATH_INFO: `/v3/users/406419fa64574de3ab8558a29af3afe8` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:06:36.666243 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1263a975-cc84-43dc-90ed-191d45727dcd None admin] RBAC: Authorizing `identity:get_user(user_id=406419fa64574de3ab8558a29af3afe8)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:06:36.667746 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1263a975-cc84-43dc-90ed-191d45727dcd None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:06:36.669442 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 597/1182] 10.222.0.26 () {56 vars in 1177 bytes} [Thu Jul 14 09:06:36 2022] GET /identity/v3/users/406419fa64574de3ab8558a29af3afe8 => generated 311 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:06:36.675736 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fb94f4-c9cc-4cba-ba66-5f0e1a99fcb0 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:06:36.689990 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:06:36.804965 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fb94f4-c9cc-4cba-ba66-5f0e1a99fcb0 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:06:36.840282 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7081cf62-ba77-4386-b5d3-a048adad65c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:06:36.840829 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7081cf62-ba77-4386-b5d3-a048adad65c6 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:06:36.841143 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7081cf62-ba77-4386-b5d3-a048adad65c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:06:36.841413 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7081cf62-ba77-4386-b5d3-a048adad65c6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:06:36.908964 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7081cf62-ba77-4386-b5d3-a048adad65c6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:06:36.910399 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7081cf62-ba77-4386-b5d3-a048adad65c6 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:06:36.921862 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:06:37.017520 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 586/1183] 10.222.0.26 () {58 vars in 1282 bytes} [Thu Jul 14 09:06:36 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 09:06:37.027084 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1263a975-cc84-43dc-90ed-191d45727dcd None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:06:37.150388 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1263a975-cc84-43dc-90ed-191d45727dcd None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:06:37.175861 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:06:37.186187 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0e28f8-d576-47eb-82c7-5ea6b5e13f08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:06:37.187045 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0e28f8-d576-47eb-82c7-5ea6b5e13f08 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:06:37.187349 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0e28f8-d576-47eb-82c7-5ea6b5e13f08 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:06:37.187621 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0e28f8-d576-47eb-82c7-5ea6b5e13f08 None admin] PATH_INFO: `/v3/projects/bfbeab0b6a0e4b27a99198dc69814320` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:06:37.188857 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0e28f8-d576-47eb-82c7-5ea6b5e13f08 None admin] RBAC: Authorizing `identity:get_project(project_id=bfbeab0b6a0e4b27a99198dc69814320)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:06:37.190286 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0e28f8-d576-47eb-82c7-5ea6b5e13f08 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:06:37.192037 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 598/1184] 10.222.0.26 () {56 vars in 1186 bytes} [Thu Jul 14 09:06:37 2022] GET /identity/v3/projects/bfbeab0b6a0e4b27a99198dc69814320 => generated 361 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:06:37.202163 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7081cf62-ba77-4386-b5d3-a048adad65c6 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:06:37.302023 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7081cf62-ba77-4386-b5d3-a048adad65c6 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:06:37.313474 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:06:37.341390 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a869bc6-173f-4010-9e70-53ab49a23dfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:06:37.342036 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a869bc6-173f-4010-9e70-53ab49a23dfe None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:06:37.342314 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a869bc6-173f-4010-9e70-53ab49a23dfe None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:06:37.342591 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a869bc6-173f-4010-9e70-53ab49a23dfe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:06:37.343458 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a869bc6-173f-4010-9e70-53ab49a23dfe None admin] RBAC: Adding query filter params (role.id=7b549d58d1724395b343609d895c80d0, scope.project.id=bfbeab0b6a0e4b27a99198dc69814320, user.id=406419fa64574de3ab8558a29af3afe8) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:06:37.343767 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a869bc6-173f-4010-9e70-53ab49a23dfe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:06:37.345195 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a869bc6-173f-4010-9e70-53ab49a23dfe None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:06:37.351183 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 587/1185] 10.222.0.26 () {56 vars in 1374 bytes} [Thu Jul 14 09:06:37 2022] GET /identity/v3/role_assignments?user.id=406419fa64574de3ab8558a29af3afe8&scope.project.id=bfbeab0b6a0e4b27a99198dc69814320&role.id=7b549d58d1724395b343609d895c80d0 => generated 259 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:06:39.320752 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afba6b91-1c2d-42d5-b83e-3b9396eb2b39 None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:06:39.321307 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afba6b91-1c2d-42d5-b83e-3b9396eb2b39 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:06:39.321307 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afba6b91-1c2d-42d5-b83e-3b9396eb2b39 None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:06:39.322672 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 599/1186] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:06:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:06:39.329091 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343b189a-b7a4-4215-be30-e47d123e2a59 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:06:39.329805 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343b189a-b7a4-4215-be30-e47d123e2a59 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:06:39.330205 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343b189a-b7a4-4215-be30-e47d123e2a59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:06:39.351512 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-343b189a-b7a4-4215-be30-e47d123e2a59 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:06:39.352707 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-343b189a-b7a4-4215-be30-e47d123e2a59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:06:39.425566 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 588/1187] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:06:39 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 09:06:39.461546 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afba6b91-1c2d-42d5-b83e-3b9396eb2b39 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:06:39.572970 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afba6b91-1c2d-42d5-b83e-3b9396eb2b39 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:06:39.608046 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd21335-15dc-4a9a-9355-3eba154dcd61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:06:39.608833 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd21335-15dc-4a9a-9355-3eba154dcd61 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:06:39.609108 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd21335-15dc-4a9a-9355-3eba154dcd61 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:06:39.609378 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd21335-15dc-4a9a-9355-3eba154dcd61 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:06:39.620280 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd21335-15dc-4a9a-9355-3eba154dcd61 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 09:06:39.627835 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-cbd21335-15dc-4a9a-9355-3eba154dcd61 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 09:06:39.627835 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:06:39.627835 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:06:39.627835 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:06:39.627835 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:06:39.627835 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:06:39.627835 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:06:39.627835 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:06:39.627835 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:06:39.627835 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:06:39.627835 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:06:39.627835 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:06:39.627835 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 09:06:39.627835 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 09:06:39.627835 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 09:06:39.627835 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 09:06:39.628896 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 09:06:39.628896 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 09:06:39.628896 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 09:06:39.628896 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 09:06:39.628896 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:06:39.628896 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:06:39.628896 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:06:39.628896 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:06:39.628896 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:06:39.628896 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:06:39.628896 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:06:39.628896 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 14 09:06:39.628896 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:06:39.628896 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:06:39.628896 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:06:39.628896 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:06:39.629955 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:06:39.629955 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:06:39.629955 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:06:39.629955 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:06:39.629955 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 09:06:39.629955 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 09:06:39.629955 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 09:06:39.629955 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 09:06:39.629955 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 09:06:39.629955 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 09:06:39.629955 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 09:06:39.629955 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 14 09:06:39.629955 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 600/1188] 10.222.0.26 () {56 vars in 1096 bytes} [Thu Jul 14 09:06:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:06:39.636393 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343b189a-b7a4-4215-be30-e47d123e2a59 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:06:39.751554 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343b189a-b7a4-4215-be30-e47d123e2a59 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:06:39.753912 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:06:39.786572 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0caeea3a-f35b-4805-90f0-bb8bef43b656 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:06:39.787232 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caeea3a-f35b-4805-90f0-bb8bef43b656 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:06:39.787546 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caeea3a-f35b-4805-90f0-bb8bef43b656 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:06:39.787822 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caeea3a-f35b-4805-90f0-bb8bef43b656 None admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:06:39.788575 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0caeea3a-f35b-4805-90f0-bb8bef43b656 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:06:39.788935 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0caeea3a-f35b-4805-90f0-bb8bef43b656 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:06:39.790321 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0caeea3a-f35b-4805-90f0-bb8bef43b656 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:06:39.798608 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 589/1189] 10.222.0.26 () {56 vars in 1099 bytes} [Thu Jul 14 09:06:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 09:06:39.807432 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd21335-15dc-4a9a-9355-3eba154dcd61 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:06:39.866784 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:06:39.914031 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd21335-15dc-4a9a-9355-3eba154dcd61 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:06:39.949672 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e6ee58-74c1-4214-983d-5158ac63bfdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:06:39.950531 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e6ee58-74c1-4214-983d-5158ac63bfdb None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:06:39.950808 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e6ee58-74c1-4214-983d-5158ac63bfdb None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:06:39.951079 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e6ee58-74c1-4214-983d-5158ac63bfdb None admin] PATH_INFO: `/v3/users/406419fa64574de3ab8558a29af3afe8` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:06:39.952429 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e6ee58-74c1-4214-983d-5158ac63bfdb None admin] RBAC: Authorizing `identity:get_user(user_id=406419fa64574de3ab8558a29af3afe8)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:06:39.953867 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e6ee58-74c1-4214-983d-5158ac63bfdb None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:06:39.955552 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 601/1190] 10.222.0.26 () {56 vars in 1177 bytes} [Thu Jul 14 09:06:39 2022] GET /identity/v3/users/406419fa64574de3ab8558a29af3afe8 => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:06:39.963959 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0caeea3a-f35b-4805-90f0-bb8bef43b656 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:06:40.071295 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0caeea3a-f35b-4805-90f0-bb8bef43b656 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:06:40.108008 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0ea109-9e97-4295-ba37-62bc23fbb907 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:06:40.108853 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0ea109-9e97-4295-ba37-62bc23fbb907 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:06:40.109137 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0ea109-9e97-4295-ba37-62bc23fbb907 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:06:40.109416 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0ea109-9e97-4295-ba37-62bc23fbb907 None admin] PATH_INFO: `/v3/projects/bfbeab0b6a0e4b27a99198dc69814320` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:06:40.110629 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0ea109-9e97-4295-ba37-62bc23fbb907 None admin] RBAC: Authorizing `identity:get_project(project_id=bfbeab0b6a0e4b27a99198dc69814320)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:06:40.112098 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0ea109-9e97-4295-ba37-62bc23fbb907 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:06:40.113804 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 590/1191] 10.222.0.26 () {56 vars in 1186 bytes} [Thu Jul 14 09:06:39 2022] GET /identity/v3/projects/bfbeab0b6a0e4b27a99198dc69814320 => generated 361 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:06:40.120508 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e6ee58-74c1-4214-983d-5158ac63bfdb None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:06:40.234723 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e6ee58-74c1-4214-983d-5158ac63bfdb None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:06:40.256911 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:06:40.269791 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3eae18d-b639-4f3e-9739-a4583c8deec3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:06:40.270659 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3eae18d-b639-4f3e-9739-a4583c8deec3 None admin] REQUEST_METHOD: `PUT` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:06:40.270937 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3eae18d-b639-4f3e-9739-a4583c8deec3 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:06:40.271214 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3eae18d-b639-4f3e-9739-a4583c8deec3 None admin] PATH_INFO: `/v3/projects/bfbeab0b6a0e4b27a99198dc69814320/users/406419fa64574de3ab8558a29af3afe8/roles/7b549d58d1724395b343609d895c80d0` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:06:40.273487 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3eae18d-b639-4f3e-9739-a4583c8deec3 None admin] RBAC: Authorizing `identity:create_grant(project_id=bfbeab0b6a0e4b27a99198dc69814320, user_id=406419fa64574de3ab8558a29af3afe8, role_id=7b549d58d1724395b343609d895c80d0)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:06:40.274976 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3eae18d-b639-4f3e-9739-a4583c8deec3 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:06:40.290498 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 602/1192] 10.222.0.26 () {58 vars in 1439 bytes} [Thu Jul 14 09:06:40 2022] PUT /identity/v3/projects/bfbeab0b6a0e4b27a99198dc69814320/users/406419fa64574de3ab8558a29af3afe8/roles/7b549d58d1724395b343609d895c80d0 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:06:42.258076 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fcc013-ec1f-4ea6-9cc8-8904563127bd None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:06:42.258959 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fcc013-ec1f-4ea6-9cc8-8904563127bd None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:06:42.258959 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fcc013-ec1f-4ea6-9cc8-8904563127bd None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:06:42.261226 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 591/1193] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:06:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:06:42.267473 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3fd044-1e25-4951-bcc4-2fdbce1547ce None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:06:42.268188 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3fd044-1e25-4951-bcc4-2fdbce1547ce None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:06:42.268340 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3fd044-1e25-4951-bcc4-2fdbce1547ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:06:42.289710 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-db3fd044-1e25-4951-bcc4-2fdbce1547ce None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:06:42.290901 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-db3fd044-1e25-4951-bcc4-2fdbce1547ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:06:42.374031 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 603/1194] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:06:42 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 09:06:42.408688 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fcc013-ec1f-4ea6-9cc8-8904563127bd None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:06:42.454943 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:06:42.528460 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fcc013-ec1f-4ea6-9cc8-8904563127bd None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:06:42.564699 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e76130-70f9-40c5-bb2e-18eac89e8327 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:06:42.565494 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e76130-70f9-40c5-bb2e-18eac89e8327 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:06:42.565779 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e76130-70f9-40c5-bb2e-18eac89e8327 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:06:42.566047 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e76130-70f9-40c5-bb2e-18eac89e8327 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:06:42.576731 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e76130-70f9-40c5-bb2e-18eac89e8327 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 09:06:42.584380 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-84e76130-70f9-40c5-bb2e-18eac89e8327 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 09:06:42.584380 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:06:42.584380 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:06:42.584380 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:06:42.584380 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:06:42.584380 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:06:42.584380 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:06:42.584380 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:06:42.584380 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:06:42.584380 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:06:42.584380 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:06:42.584380 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:06:42.584380 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 09:06:42.584380 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 09:06:42.584380 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 09:06:42.584380 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 09:06:42.585450 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 09:06:42.585450 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 09:06:42.585450 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 09:06:42.585450 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 09:06:42.585450 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:06:42.585450 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:06:42.585450 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:06:42.585450 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:06:42.585450 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:06:42.585450 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:06:42.585450 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:06:42.585450 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 14 09:06:42.585450 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:06:42.585450 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:06:42.585450 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:06:42.585450 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:06:42.586467 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:06:42.586467 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:06:42.586467 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:06:42.586467 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:06:42.586467 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 09:06:42.586467 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 09:06:42.586467 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 09:06:42.586467 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 09:06:42.586467 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 09:06:42.586467 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 09:06:42.586467 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 09:06:42.586467 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 14 09:06:42.586467 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 592/1195] 10.222.0.26 () {56 vars in 1096 bytes} [Thu Jul 14 09:06:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:06:42.592197 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3fd044-1e25-4951-bcc4-2fdbce1547ce None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:06:42.698844 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3fd044-1e25-4951-bcc4-2fdbce1547ce None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:06:42.712118 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:06:42.736543 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb2f71d-9859-4978-a6a6-dcd9900792f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:06:42.737200 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb2f71d-9859-4978-a6a6-dcd9900792f5 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:06:42.737481 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb2f71d-9859-4978-a6a6-dcd9900792f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:06:42.737753 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb2f71d-9859-4978-a6a6-dcd9900792f5 None admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:06:42.738502 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb2f71d-9859-4978-a6a6-dcd9900792f5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:06:42.738805 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb2f71d-9859-4978-a6a6-dcd9900792f5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:06:42.740206 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb2f71d-9859-4978-a6a6-dcd9900792f5 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:06:42.748934 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 604/1196] 10.222.0.26 () {56 vars in 1099 bytes} [Thu Jul 14 09:06:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:06:42.756607 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e76130-70f9-40c5-bb2e-18eac89e8327 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:06:42.866748 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:06:42.878894 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e76130-70f9-40c5-bb2e-18eac89e8327 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:06:42.914973 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f3261e-57bc-4fa2-adc9-b5ddcf8b68dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:06:42.915855 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f3261e-57bc-4fa2-adc9-b5ddcf8b68dd None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:06:42.916150 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f3261e-57bc-4fa2-adc9-b5ddcf8b68dd None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:06:42.916427 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f3261e-57bc-4fa2-adc9-b5ddcf8b68dd None admin] PATH_INFO: `/v3/users/406419fa64574de3ab8558a29af3afe8` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:06:42.917818 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f3261e-57bc-4fa2-adc9-b5ddcf8b68dd None admin] RBAC: Authorizing `identity:get_user(user_id=406419fa64574de3ab8558a29af3afe8)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:06:42.919295 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f3261e-57bc-4fa2-adc9-b5ddcf8b68dd None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:06:42.920931 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 593/1197] 10.222.0.26 () {56 vars in 1177 bytes} [Thu Jul 14 09:06:42 2022] GET /identity/v3/users/406419fa64574de3ab8558a29af3afe8 => generated 311 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:06:42.929874 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb2f71d-9859-4978-a6a6-dcd9900792f5 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:06:43.037262 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb2f71d-9859-4978-a6a6-dcd9900792f5 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:06:43.075053 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06957e9f-7d46-4ef5-ac4f-0dfe42b366b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:06:43.075718 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06957e9f-7d46-4ef5-ac4f-0dfe42b366b4 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:06:43.076006 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06957e9f-7d46-4ef5-ac4f-0dfe42b366b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:06:43.076279 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06957e9f-7d46-4ef5-ac4f-0dfe42b366b4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:06:43.119241 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:06:43.147089 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06957e9f-7d46-4ef5-ac4f-0dfe42b366b4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:06:43.148568 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06957e9f-7d46-4ef5-ac4f-0dfe42b366b4 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:06:43.248388 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 605/1198] 10.222.0.26 () {58 vars in 1282 bytes} [Thu Jul 14 09:06:42 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 09:06:43.254552 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f3261e-57bc-4fa2-adc9-b5ddcf8b68dd None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:06:43.367775 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f3261e-57bc-4fa2-adc9-b5ddcf8b68dd None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:06:43.402453 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebbd953-497d-459c-9715-6843574972b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:06:43.403238 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebbd953-497d-459c-9715-6843574972b7 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:06:43.403544 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebbd953-497d-459c-9715-6843574972b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:06:43.403785 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebbd953-497d-459c-9715-6843574972b7 None admin] PATH_INFO: `/v3/projects/bfbeab0b6a0e4b27a99198dc69814320` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:06:43.405113 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebbd953-497d-459c-9715-6843574972b7 None admin] RBAC: Authorizing `identity:get_project(project_id=bfbeab0b6a0e4b27a99198dc69814320)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:06:43.406545 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebbd953-497d-459c-9715-6843574972b7 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:06:43.408343 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 594/1199] 10.222.0.26 () {56 vars in 1186 bytes} [Thu Jul 14 09:06:43 2022] GET /identity/v3/projects/bfbeab0b6a0e4b27a99198dc69814320 => generated 361 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:06:43.417747 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06957e9f-7d46-4ef5-ac4f-0dfe42b366b4 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:06:43.507653 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:06:43.536352 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06957e9f-7d46-4ef5-ac4f-0dfe42b366b4 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:06:43.571856 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e573290-edd5-44ff-b408-dff93cc02192 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:06:43.572510 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e573290-edd5-44ff-b408-dff93cc02192 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:06:43.572786 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e573290-edd5-44ff-b408-dff93cc02192 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:06:43.573057 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e573290-edd5-44ff-b408-dff93cc02192 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:06:43.573935 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e573290-edd5-44ff-b408-dff93cc02192 None admin] RBAC: Adding query filter params (role.id=7b549d58d1724395b343609d895c80d0, scope.project.id=bfbeab0b6a0e4b27a99198dc69814320, user.id=406419fa64574de3ab8558a29af3afe8) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:06:43.574237 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e573290-edd5-44ff-b408-dff93cc02192 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:06:43.575695 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e573290-edd5-44ff-b408-dff93cc02192 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:06:43.581600 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 606/1200] 10.222.0.26 () {56 vars in 1374 bytes} [Thu Jul 14 09:06:43 2022] GET /identity/v3/role_assignments?user.id=406419fa64574de3ab8558a29af3afe8&scope.project.id=bfbeab0b6a0e4b27a99198dc69814320&role.id=7b549d58d1724395b343609d895c80d0 => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:06:53.397468 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=85 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:06:53.578915 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=31 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:07:38.379488 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba215849-199f-4889-833c-831955f8daaa None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:07:38.380187 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba215849-199f-4889-833c-831955f8daaa None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:07:38.380187 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba215849-199f-4889-833c-831955f8daaa None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:07:38.381912 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 595/1201] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:07:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 09:07:38.388482 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8309988b-0744-4bdf-a372-7e8a42bda1f0 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:07:38.389044 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8309988b-0744-4bdf-a372-7e8a42bda1f0 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:07:38.389044 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8309988b-0744-4bdf-a372-7e8a42bda1f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:07:38.408087 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-8309988b-0744-4bdf-a372-7e8a42bda1f0 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:07:38.409746 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-8309988b-0744-4bdf-a372-7e8a42bda1f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:07:38.498893 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 607/1202] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:07:38 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 09:07:38.525108 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba215849-199f-4889-833c-831955f8daaa None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:07:38.676598 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba215849-199f-4889-833c-831955f8daaa None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:07:38.717773 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6362bb79-d9ab-472f-aa03-207c605c0487 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:07:38.718580 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6362bb79-d9ab-472f-aa03-207c605c0487 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:07:38.718860 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6362bb79-d9ab-472f-aa03-207c605c0487 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:07:38.719132 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6362bb79-d9ab-472f-aa03-207c605c0487 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:07:38.734281 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6362bb79-d9ab-472f-aa03-207c605c0487 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:07:38.735837 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6362bb79-d9ab-472f-aa03-207c605c0487 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:07:38.760973 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-6362bb79-d9ab-472f-aa03-207c605c0487 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 09:07:38.760973 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:07:38.760973 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:07:38.760973 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:07:38.760973 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:07:38.760973 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:07:38.760973 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:07:38.760973 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:07:38.760973 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:07:38.760973 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:07:38.760973 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:07:38.760973 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:07:38.760973 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 09:07:38.760973 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 09:07:38.760973 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 09:07:38.762353 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 09:07:38.762353 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:07:38.762353 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:07:38.762353 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:07:38.762353 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:07:38.762353 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:07:38.762353 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:07:38.762353 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:07:38.762353 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 14 09:07:38.762353 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:07:38.762353 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:07:38.762353 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:07:38.762353 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:07:38.762353 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:07:38.762353 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:07:38.762353 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:07:38.762353 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:07:38.763822 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 09:07:38.763822 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 09:07:38.763822 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 09:07:38.763822 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 14 09:07:38.765611 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 596/1203] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jul 14 09:07:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 09:07:38.772489 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8309988b-0744-4bdf-a372-7e8a42bda1f0 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:07:38.897040 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8309988b-0744-4bdf-a372-7e8a42bda1f0 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:07:38.943173 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:07:38.952048 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e043266c-14f8-4419-b5fb-312141e14753 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:07:38.952728 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e043266c-14f8-4419-b5fb-312141e14753 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:07:38.953015 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e043266c-14f8-4419-b5fb-312141e14753 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:07:38.953297 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e043266c-14f8-4419-b5fb-312141e14753 None admin] PATH_INFO: `/v3/domains` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:07:38.954163 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e043266c-14f8-4419-b5fb-312141e14753 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:07:38.954398 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e043266c-14f8-4419-b5fb-312141e14753 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:07:38.955959 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e043266c-14f8-4419-b5fb-312141e14753 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:07:38.968912 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 608/1204] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Jul 14 09:07:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 09:07:38.975475 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6362bb79-d9ab-472f-aa03-207c605c0487 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:07:39.057649 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:07:39.136162 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6362bb79-d9ab-472f-aa03-207c605c0487 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:07:39.181574 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dcac58-c893-4067-ae98-50d8e4b6abfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:07:39.182254 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dcac58-c893-4067-ae98-50d8e4b6abfe None admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:07:39.182533 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dcac58-c893-4067-ae98-50d8e4b6abfe None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:07:39.182801 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dcac58-c893-4067-ae98-50d8e4b6abfe None admin] PATH_INFO: `/v3/users` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:07:39.183595 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74dcac58-c893-4067-ae98-50d8e4b6abfe None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:07:39.185043 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74dcac58-c893-4067-ae98-50d8e4b6abfe None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:07:39.215008 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 597/1205] 10.222.0.26 () {60 vars in 1132 bytes} [Thu Jul 14 09:07:38 2022] POST /identity/v3/users => generated 251 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 09:07:41.499878 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc337540-ce2b-42f7-9eda-b3f8de58462e None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:07:41.500447 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc337540-ce2b-42f7-9eda-b3f8de58462e None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:07:41.500447 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc337540-ce2b-42f7-9eda-b3f8de58462e None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:07:41.501809 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 609/1206] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:07:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:07:41.507509 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43880b59-6b23-42c1-94a6-721db4928469 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:07:41.508006 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43880b59-6b23-42c1-94a6-721db4928469 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:07:41.508154 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43880b59-6b23-42c1-94a6-721db4928469 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:07:41.527413 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-43880b59-6b23-42c1-94a6-721db4928469 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:07:41.528854 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-43880b59-6b23-42c1-94a6-721db4928469 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:07:41.605411 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 598/1207] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:07:41 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 09:07:41.630760 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc337540-ce2b-42f7-9eda-b3f8de58462e None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:07:41.748377 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc337540-ce2b-42f7-9eda-b3f8de58462e None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:07:41.785193 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422d9f88-64b8-4432-adcb-f4b0996ba04b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:07:41.785925 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422d9f88-64b8-4432-adcb-f4b0996ba04b None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:07:41.787562 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422d9f88-64b8-4432-adcb-f4b0996ba04b None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:07:41.787849 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422d9f88-64b8-4432-adcb-f4b0996ba04b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:07:41.799743 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422d9f88-64b8-4432-adcb-f4b0996ba04b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 09:07:41.808299 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-422d9f88-64b8-4432-adcb-f4b0996ba04b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 09:07:41.808299 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:07:41.808299 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:07:41.808299 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:07:41.808299 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:07:41.808299 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:07:41.808299 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:07:41.808299 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:07:41.808299 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:07:41.808299 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:07:41.808299 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:07:41.808299 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:07:41.808299 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 09:07:41.808299 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 09:07:41.808299 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 09:07:41.808299 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 09:07:41.809603 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 09:07:41.809603 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 09:07:41.809603 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 09:07:41.809603 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 09:07:41.809603 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:07:41.809603 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:07:41.809603 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:07:41.809603 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:07:41.809603 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:07:41.809603 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:07:41.809603 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:07:41.809603 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 14 09:07:41.809603 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:07:41.809603 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:07:41.809603 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:07:41.809603 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:07:41.810587 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:07:41.810587 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:07:41.810587 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:07:41.810587 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:07:41.810587 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 09:07:41.810587 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 09:07:41.810587 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 09:07:41.810587 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 09:07:41.810587 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 09:07:41.810587 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 09:07:41.810587 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 09:07:41.810587 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 14 09:07:41.810587 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 610/1208] 10.222.0.26 () {56 vars in 1102 bytes} [Thu Jul 14 09:07:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:07:41.815846 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43880b59-6b23-42c1-94a6-721db4928469 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:07:41.860303 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:07:41.929682 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43880b59-6b23-42c1-94a6-721db4928469 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:07:41.966704 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d504a951-d345-42fd-b402-1c4e9286ac3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:07:41.967552 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d504a951-d345-42fd-b402-1c4e9286ac3c None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:07:41.967931 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d504a951-d345-42fd-b402-1c4e9286ac3c None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:07:41.968321 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d504a951-d345-42fd-b402-1c4e9286ac3c None admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:07:41.969168 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d504a951-d345-42fd-b402-1c4e9286ac3c None admin] RBAC: Adding query filter params (name=service) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:07:41.969583 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d504a951-d345-42fd-b402-1c4e9286ac3c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:07:41.971033 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d504a951-d345-42fd-b402-1c4e9286ac3c None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:07:41.981528 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 599/1209] 10.222.0.26 () {56 vars in 1103 bytes} [Thu Jul 14 09:07:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:07:41.988119 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422d9f88-64b8-4432-adcb-f4b0996ba04b None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:07:42.053387 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:07:42.108925 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422d9f88-64b8-4432-adcb-f4b0996ba04b None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:07:42.146881 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06eb83fb-cfee-4483-9de6-46e1a27adda3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:07:42.147726 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06eb83fb-cfee-4483-9de6-46e1a27adda3 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:07:42.148101 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06eb83fb-cfee-4483-9de6-46e1a27adda3 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:07:42.148278 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06eb83fb-cfee-4483-9de6-46e1a27adda3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:07:42.160085 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06eb83fb-cfee-4483-9de6-46e1a27adda3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:07:42.161878 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06eb83fb-cfee-4483-9de6-46e1a27adda3 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:07:42.182291 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-06eb83fb-cfee-4483-9de6-46e1a27adda3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 09:07:42.182291 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:07:42.182291 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:07:42.182291 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:07:42.182291 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:07:42.182291 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:07:42.182291 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:07:42.182291 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:07:42.182291 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:07:42.182291 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:07:42.182291 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:07:42.182291 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:07:42.182291 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 09:07:42.182291 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 09:07:42.182291 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 09:07:42.189824 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 09:07:42.189824 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:07:42.189824 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:07:42.189824 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:07:42.189824 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:07:42.189824 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:07:42.189824 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:07:42.189824 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:07:42.189824 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 14 09:07:42.189824 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:07:42.189824 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:07:42.189824 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:07:42.189824 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:07:42.189824 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:07:42.189824 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:07:42.189824 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:07:42.189824 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:07:42.190952 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 09:07:42.190952 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 09:07:42.190952 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 09:07:42.190952 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 14 09:07:42.190952 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 611/1210] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jul 14 09:07:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:07:42.191349 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d504a951-d345-42fd-b402-1c4e9286ac3c None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:07:42.302877 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:07:42.305911 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d504a951-d345-42fd-b402-1c4e9286ac3c None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:07:42.352312 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02dc2a72-2c85-4f09-bb53-74896160d7d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:07:42.353275 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dc2a72-2c85-4f09-bb53-74896160d7d8 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:07:42.353804 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dc2a72-2c85-4f09-bb53-74896160d7d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:07:42.354238 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dc2a72-2c85-4f09-bb53-74896160d7d8 None admin] PATH_INFO: `/v3/domains` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:07:42.356411 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02dc2a72-2c85-4f09-bb53-74896160d7d8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:07:42.356828 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02dc2a72-2c85-4f09-bb53-74896160d7d8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:07:42.359499 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02dc2a72-2c85-4f09-bb53-74896160d7d8 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:07:42.381573 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 600/1211] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Jul 14 09:07:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:07:42.383147 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06eb83fb-cfee-4483-9de6-46e1a27adda3 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:07:42.516383 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06eb83fb-cfee-4483-9de6-46e1a27adda3 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:07:42.518888 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:07:42.566788 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dca741-31d2-4b7f-abd3-1121b2ee77c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:07:42.566788 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dca741-31d2-4b7f-abd3-1121b2ee77c1 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:07:42.566788 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dca741-31d2-4b7f-abd3-1121b2ee77c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:07:42.566788 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dca741-31d2-4b7f-abd3-1121b2ee77c1 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:07:42.576596 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49dca741-31d2-4b7f-abd3-1121b2ee77c1 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:07:42.578237 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49dca741-31d2-4b7f-abd3-1121b2ee77c1 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:07:42.598025 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-49dca741-31d2-4b7f-abd3-1121b2ee77c1 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 14 09:07:42.598025 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:07:42.598025 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:07:42.598025 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:07:42.598025 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:07:42.598025 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:07:42.598025 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:07:42.598025 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:07:42.598025 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:07:42.598025 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:07:42.598025 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:07:42.598025 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:07:42.598025 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 09:07:42.598025 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 09:07:42.598025 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 09:07:42.598025 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 09:07:42.599251 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:07:42.599251 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:07:42.599251 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 09:07:42.599251 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 09:07:42.599251 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 09:07:42.599251 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 09:07:42.599251 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:07:42.599251 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:07:42.599251 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:07:42.599251 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:07:42.599251 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:07:42.599251 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 14 09:07:42.599251 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:07:42.599251 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:07:42.599251 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:07:42.599251 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:07:42.599251 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:07:42.599251 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:07:42.600316 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:07:42.600316 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:07:42.600316 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 09:07:42.600316 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 09:07:42.600316 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 09:07:42.600316 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 09:07:42.600316 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 09:07:42.600316 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 09:07:42.600316 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 14 09:07:42.600316 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 14 09:07:42.602626 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 612/1212] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jul 14 09:07:42 2022] GET /identity/v3/users/placement => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:07:42.608427 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02dc2a72-2c85-4f09-bb53-74896160d7d8 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:07:42.742327 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02dc2a72-2c85-4f09-bb53-74896160d7d8 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:07:42.784334 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9faf2e-3541-446f-8f3a-5f9c0ceea7ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:07:42.785179 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9faf2e-3541-446f-8f3a-5f9c0ceea7ea None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:07:42.785635 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9faf2e-3541-446f-8f3a-5f9c0ceea7ea None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:07:42.785952 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9faf2e-3541-446f-8f3a-5f9c0ceea7ea None admin] PATH_INFO: `/v3/users` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:07:42.786905 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9faf2e-3541-446f-8f3a-5f9c0ceea7ea None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:07:42.787243 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9faf2e-3541-446f-8f3a-5f9c0ceea7ea None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:07:42.789081 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9faf2e-3541-446f-8f3a-5f9c0ceea7ea None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:07:42.814346 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 601/1213] 10.222.0.26 () {56 vars in 1143 bytes} [Thu Jul 14 09:07:42 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:07:42.822685 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dca741-31d2-4b7f-abd3-1121b2ee77c1 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:07:42.966333 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dca741-31d2-4b7f-abd3-1121b2ee77c1 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:07:43.016889 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3ff7b2-4034-4b6f-8309-93f19553b36f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:07:43.018219 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3ff7b2-4034-4b6f-8309-93f19553b36f None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:07:43.018506 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3ff7b2-4034-4b6f-8309-93f19553b36f None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:07:43.018778 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3ff7b2-4034-4b6f-8309-93f19553b36f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:07:43.068364 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:07:43.118618 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3ff7b2-4034-4b6f-8309-93f19553b36f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:07:43.120348 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3ff7b2-4034-4b6f-8309-93f19553b36f None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:07:43.241580 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 613/1214] 10.222.0.26 () {58 vars in 1282 bytes} [Thu Jul 14 09:07:42 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 09:07:43.248141 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9faf2e-3541-446f-8f3a-5f9c0ceea7ea None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:07:43.281994 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:07:43.403233 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9faf2e-3541-446f-8f3a-5f9c0ceea7ea None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:07:43.448401 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55faf1dd-2bb3-4b7e-941f-10f2ff977acb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:07:43.449265 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55faf1dd-2bb3-4b7e-941f-10f2ff977acb None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:07:43.449509 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55faf1dd-2bb3-4b7e-941f-10f2ff977acb None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:07:43.449788 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55faf1dd-2bb3-4b7e-941f-10f2ff977acb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:07:43.465087 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55faf1dd-2bb3-4b7e-941f-10f2ff977acb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:07:43.466578 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55faf1dd-2bb3-4b7e-941f-10f2ff977acb None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:07:43.489439 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-55faf1dd-2bb3-4b7e-941f-10f2ff977acb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 09:07:43.489439 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:07:43.489439 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:07:43.489439 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:07:43.489439 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:07:43.489439 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:07:43.489439 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:07:43.489439 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:07:43.489439 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:07:43.489439 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:07:43.489439 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:07:43.489439 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:07:43.489439 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 09:07:43.489439 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 09:07:43.489439 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 09:07:43.490935 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 09:07:43.490935 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:07:43.490935 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:07:43.490935 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:07:43.490935 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:07:43.490935 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:07:43.490935 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:07:43.490935 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:07:43.490935 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 14 09:07:43.490935 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:07:43.490935 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:07:43.490935 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:07:43.490935 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:07:43.490935 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:07:43.490935 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:07:43.490935 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:07:43.490935 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:07:43.491947 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 09:07:43.491947 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 09:07:43.491947 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 09:07:43.491947 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 14 09:07:43.491947 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 602/1215] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jul 14 09:07:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:07:43.498625 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3ff7b2-4034-4b6f-8309-93f19553b36f None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:07:43.568629 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:07:43.609350 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3ff7b2-4034-4b6f-8309-93f19553b36f None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:07:43.645780 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548c885e-5138-4e66-a14d-f026dd69c076 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:07:43.646462 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548c885e-5138-4e66-a14d-f026dd69c076 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:07:43.646743 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548c885e-5138-4e66-a14d-f026dd69c076 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:07:43.647019 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548c885e-5138-4e66-a14d-f026dd69c076 None admin] PATH_INFO: `/v3/domains` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:07:43.647809 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548c885e-5138-4e66-a14d-f026dd69c076 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:07:43.648122 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548c885e-5138-4e66-a14d-f026dd69c076 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:07:43.649503 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548c885e-5138-4e66-a14d-f026dd69c076 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:07:43.661792 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 614/1216] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Jul 14 09:07:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:07:43.667571 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55faf1dd-2bb3-4b7e-941f-10f2ff977acb None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:07:43.759967 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:07:43.801559 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55faf1dd-2bb3-4b7e-941f-10f2ff977acb None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:07:43.838445 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6436e4f2-cc65-4a91-a82d-5398e12294ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:07:43.839279 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6436e4f2-cc65-4a91-a82d-5398e12294ef None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:07:43.839566 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6436e4f2-cc65-4a91-a82d-5398e12294ef None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:07:43.839842 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6436e4f2-cc65-4a91-a82d-5398e12294ef None admin] PATH_INFO: `/v3/projects/service` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:07:43.846777 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6436e4f2-cc65-4a91-a82d-5398e12294ef None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:07:43.848257 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6436e4f2-cc65-4a91-a82d-5398e12294ef None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:07:43.863174 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-6436e4f2-cc65-4a91-a82d-5398e12294ef None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 09:07:43.863174 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:07:43.863174 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:07:43.863174 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:07:43.863174 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:07:43.863174 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:07:43.863174 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:07:43.863174 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:07:43.863174 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:07:43.863174 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:07:43.863174 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:07:43.863174 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:07:43.863174 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 09:07:43.863174 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 09:07:43.863174 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 09:07:43.864373 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 09:07:43.864373 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:07:43.864373 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:07:43.864373 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:07:43.864373 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:07:43.864373 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:07:43.864373 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:07:43.864373 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:07:43.864373 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 14 09:07:43.864373 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:07:43.864373 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:07:43.864373 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:07:43.864373 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:07:43.864373 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:07:43.864373 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:07:43.864373 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:07:43.864373 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:07:43.865517 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 09:07:43.865517 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 09:07:43.865517 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 09:07:43.865517 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 09:07:43.865517 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 09:07:43.865517 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 09:07:43.865517 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 09:07:43.865517 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 14 09:07:43.869735 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 603/1217] 10.222.0.26 () {56 vars in 1111 bytes} [Thu Jul 14 09:07:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:07:43.871111 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548c885e-5138-4e66-a14d-f026dd69c076 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:07:43.983112 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548c885e-5138-4e66-a14d-f026dd69c076 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:07:44.009067 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:07:44.025019 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cff628-ef36-48e7-9757-2b2d93149ac8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:07:44.025977 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cff628-ef36-48e7-9757-2b2d93149ac8 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:07:44.026498 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cff628-ef36-48e7-9757-2b2d93149ac8 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:07:44.026902 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cff628-ef36-48e7-9757-2b2d93149ac8 None admin] PATH_INFO: `/v3/projects` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:07:44.027927 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cff628-ef36-48e7-9757-2b2d93149ac8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:07:44.028390 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cff628-ef36-48e7-9757-2b2d93149ac8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:07:44.030496 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cff628-ef36-48e7-9757-2b2d93149ac8 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:07:44.052966 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 615/1218] 10.222.0.26 () {56 vars in 1148 bytes} [Thu Jul 14 09:07:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 09:07:44.058909 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6436e4f2-cc65-4a91-a82d-5398e12294ef None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:07:44.177090 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6436e4f2-cc65-4a91-a82d-5398e12294ef None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:07:44.209789 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:07:44.217352 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2c49da-f125-4ab0-bcb4-db36a92a3e8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:07:44.218081 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2c49da-f125-4ab0-bcb4-db36a92a3e8a None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:07:44.218364 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2c49da-f125-4ab0-bcb4-db36a92a3e8a None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:07:44.218636 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2c49da-f125-4ab0-bcb4-db36a92a3e8a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:07:44.219514 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2c49da-f125-4ab0-bcb4-db36a92a3e8a None admin] RBAC: Adding query filter params (role.id=f5c487bc605f45df82ef6367c80fae70, scope.project.id=33bf210d015847439655b9943d1b0261, user.id=355684f52e074ad9bfa7fd90bcae6caf) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:07:44.219963 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2c49da-f125-4ab0-bcb4-db36a92a3e8a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:07:44.221434 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2c49da-f125-4ab0-bcb4-db36a92a3e8a None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:07:44.227311 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 604/1219] 10.222.0.26 () {56 vars in 1374 bytes} [Thu Jul 14 09:07:44 2022] GET /identity/v3/role_assignments?user.id=355684f52e074ad9bfa7fd90bcae6caf&scope.project.id=33bf210d015847439655b9943d1b0261&role.id=f5c487bc605f45df82ef6367c80fae70 => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:07:46.276840 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff79b3c-5342-48d9-b265-e24c87540e23 None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:07:46.277467 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff79b3c-5342-48d9-b265-e24c87540e23 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:07:46.277467 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff79b3c-5342-48d9-b265-e24c87540e23 None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:07:46.278891 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 616/1220] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:07:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:07:46.284238 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcf50b1-4c00-48db-ab72-9d58e49e387e None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:07:46.284695 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcf50b1-4c00-48db-ab72-9d58e49e387e None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:07:46.284789 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcf50b1-4c00-48db-ab72-9d58e49e387e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:07:46.303090 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-dfcf50b1-4c00-48db-ab72-9d58e49e387e None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:07:46.305826 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-dfcf50b1-4c00-48db-ab72-9d58e49e387e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:07:46.392279 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 605/1221] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:07:46 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 09:07:46.414873 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff79b3c-5342-48d9-b265-e24c87540e23 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:07:46.524465 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff79b3c-5342-48d9-b265-e24c87540e23 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:07:46.562801 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc3c45c-d3a0-4c16-b577-64c16eead933 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:07:46.563682 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc3c45c-d3a0-4c16-b577-64c16eead933 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:07:46.563969 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc3c45c-d3a0-4c16-b577-64c16eead933 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:07:46.564242 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc3c45c-d3a0-4c16-b577-64c16eead933 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:07:46.574827 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc3c45c-d3a0-4c16-b577-64c16eead933 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 09:07:46.582586 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-4fc3c45c-d3a0-4c16-b577-64c16eead933 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 09:07:46.582586 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:07:46.582586 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:07:46.582586 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:07:46.582586 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:07:46.582586 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:07:46.582586 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:07:46.582586 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:07:46.582586 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:07:46.582586 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:07:46.582586 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:07:46.582586 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:07:46.582586 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 09:07:46.582586 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 09:07:46.582586 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 09:07:46.582586 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 09:07:46.584099 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 09:07:46.584099 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 09:07:46.584099 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 09:07:46.584099 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 09:07:46.584099 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:07:46.584099 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:07:46.584099 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:07:46.584099 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:07:46.584099 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:07:46.584099 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:07:46.584099 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:07:46.584099 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 14 09:07:46.584099 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:07:46.584099 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:07:46.584099 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:07:46.584099 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:07:46.584971 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:07:46.584971 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:07:46.584971 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:07:46.584971 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:07:46.584971 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 09:07:46.584971 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 09:07:46.584971 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 09:07:46.584971 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 09:07:46.584971 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 09:07:46.584971 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 09:07:46.584971 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 09:07:46.584971 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 14 09:07:46.584971 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 617/1222] 10.222.0.26 () {56 vars in 1102 bytes} [Thu Jul 14 09:07:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:07:46.590186 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcf50b1-4c00-48db-ab72-9d58e49e387e None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:07:46.698060 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcf50b1-4c00-48db-ab72-9d58e49e387e None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:07:46.723805 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:07:46.734740 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0feaec-fca8-4395-9d30-6a91820871b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:07:46.735451 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0feaec-fca8-4395-9d30-6a91820871b5 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:07:46.735739 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0feaec-fca8-4395-9d30-6a91820871b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:07:46.736013 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0feaec-fca8-4395-9d30-6a91820871b5 None admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:07:46.736846 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0feaec-fca8-4395-9d30-6a91820871b5 None admin] RBAC: Adding query filter params (name=service) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:07:46.737224 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0feaec-fca8-4395-9d30-6a91820871b5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:07:46.738752 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0feaec-fca8-4395-9d30-6a91820871b5 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:07:46.747991 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 606/1223] 10.222.0.26 () {56 vars in 1103 bytes} [Thu Jul 14 09:07:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:07:46.753790 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc3c45c-d3a0-4c16-b577-64c16eead933 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:07:46.801606 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:07:46.858251 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc3c45c-d3a0-4c16-b577-64c16eead933 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:07:46.892407 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19530979-348e-4c6a-8d24-59596567596f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:07:46.893169 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19530979-348e-4c6a-8d24-59596567596f None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:07:46.893626 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19530979-348e-4c6a-8d24-59596567596f None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:07:46.893742 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19530979-348e-4c6a-8d24-59596567596f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:07:46.905912 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19530979-348e-4c6a-8d24-59596567596f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:07:46.907457 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19530979-348e-4c6a-8d24-59596567596f None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:07:46.924557 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-19530979-348e-4c6a-8d24-59596567596f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 09:07:46.924557 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:07:46.924557 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:07:46.924557 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:07:46.924557 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:07:46.924557 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:07:46.924557 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:07:46.924557 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:07:46.924557 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:07:46.924557 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:07:46.924557 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:07:46.924557 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:07:46.924557 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 09:07:46.924557 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 09:07:46.924557 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 09:07:46.925654 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 09:07:46.925654 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:07:46.925654 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:07:46.925654 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:07:46.925654 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:07:46.925654 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:07:46.925654 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:07:46.925654 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:07:46.925654 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 14 09:07:46.925654 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:07:46.925654 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:07:46.925654 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:07:46.925654 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:07:46.925654 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:07:46.925654 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:07:46.925654 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:07:46.925654 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:07:46.926712 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 09:07:46.926712 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 09:07:46.926712 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 09:07:46.926712 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 14 09:07:46.926712 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 618/1224] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jul 14 09:07:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 09:07:46.931856 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0feaec-fca8-4395-9d30-6a91820871b5 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:07:47.030431 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0feaec-fca8-4395-9d30-6a91820871b5 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:07:47.064669 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b04cde-a3ab-4b2a-b201-d5baec8c78a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:07:47.065270 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b04cde-a3ab-4b2a-b201-d5baec8c78a6 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:07:47.065549 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b04cde-a3ab-4b2a-b201-d5baec8c78a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:07:47.065824 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b04cde-a3ab-4b2a-b201-d5baec8c78a6 None admin] PATH_INFO: `/v3/domains` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:07:47.066547 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b04cde-a3ab-4b2a-b201-d5baec8c78a6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:07:47.066851 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b04cde-a3ab-4b2a-b201-d5baec8c78a6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:07:47.068269 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b04cde-a3ab-4b2a-b201-d5baec8c78a6 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:07:47.080373 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 607/1225] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Jul 14 09:07:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:07:47.086193 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19530979-348e-4c6a-8d24-59596567596f None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:07:47.185407 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:07:47.194804 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19530979-348e-4c6a-8d24-59596567596f None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:07:47.232110 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4258682-a0d1-4b29-bb3d-a563b515d0b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:07:47.233090 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4258682-a0d1-4b29-bb3d-a563b515d0b3 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:07:47.233570 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4258682-a0d1-4b29-bb3d-a563b515d0b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:07:47.233939 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4258682-a0d1-4b29-bb3d-a563b515d0b3 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:07:47.244944 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4258682-a0d1-4b29-bb3d-a563b515d0b3 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:07:47.246680 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4258682-a0d1-4b29-bb3d-a563b515d0b3 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:07:47.262798 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-f4258682-a0d1-4b29-bb3d-a563b515d0b3 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 14 09:07:47.262798 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:07:47.262798 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:07:47.262798 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:07:47.262798 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:07:47.262798 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:07:47.262798 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:07:47.262798 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:07:47.262798 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:07:47.262798 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:07:47.262798 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:07:47.262798 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:07:47.262798 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 09:07:47.262798 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 09:07:47.262798 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 09:07:47.262798 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 09:07:47.264017 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:07:47.264017 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:07:47.264017 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 09:07:47.264017 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 09:07:47.264017 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 09:07:47.264017 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 09:07:47.264017 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:07:47.264017 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:07:47.264017 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:07:47.264017 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:07:47.264017 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:07:47.264017 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 14 09:07:47.264017 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:07:47.264017 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:07:47.264017 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:07:47.264017 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:07:47.264017 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:07:47.264017 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:07:47.265199 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:07:47.265199 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:07:47.265199 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 09:07:47.265199 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 09:07:47.265199 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 09:07:47.265199 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 09:07:47.265199 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 09:07:47.265199 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 09:07:47.265199 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 14 09:07:47.265199 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 14 09:07:47.265199 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 619/1226] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jul 14 09:07:47 2022] GET /identity/v3/users/placement => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:07:47.270352 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b04cde-a3ab-4b2a-b201-d5baec8c78a6 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:07:47.324003 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:07:47.389609 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b04cde-a3ab-4b2a-b201-d5baec8c78a6 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:07:47.428571 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4426ee47-3f86-4bbd-bf0f-0b09220de8eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:07:47.429325 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4426ee47-3f86-4bbd-bf0f-0b09220de8eb None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:07:47.429686 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4426ee47-3f86-4bbd-bf0f-0b09220de8eb None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:07:47.429955 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4426ee47-3f86-4bbd-bf0f-0b09220de8eb None admin] PATH_INFO: `/v3/users` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:07:47.430787 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4426ee47-3f86-4bbd-bf0f-0b09220de8eb None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:07:47.431092 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4426ee47-3f86-4bbd-bf0f-0b09220de8eb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:07:47.432538 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4426ee47-3f86-4bbd-bf0f-0b09220de8eb None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:07:47.452522 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 608/1227] 10.222.0.26 () {56 vars in 1143 bytes} [Thu Jul 14 09:07:47 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:07:47.458404 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4258682-a0d1-4b29-bb3d-a563b515d0b3 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:07:47.572649 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4258682-a0d1-4b29-bb3d-a563b515d0b3 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:07:47.608637 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3625d59f-26a0-43ae-a0fd-94b77dbea601 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:07:47.609642 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3625d59f-26a0-43ae-a0fd-94b77dbea601 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:07:47.610040 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3625d59f-26a0-43ae-a0fd-94b77dbea601 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:07:47.610349 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3625d59f-26a0-43ae-a0fd-94b77dbea601 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:07:47.624166 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3625d59f-26a0-43ae-a0fd-94b77dbea601 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:07:47.625638 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3625d59f-26a0-43ae-a0fd-94b77dbea601 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:07:47.643395 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-3625d59f-26a0-43ae-a0fd-94b77dbea601 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 09:07:47.643395 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:07:47.643395 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:07:47.643395 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:07:47.643395 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:07:47.643395 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:07:47.643395 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:07:47.643395 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:07:47.643395 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:07:47.643395 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:07:47.643395 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:07:47.643395 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:07:47.643395 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 09:07:47.643395 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 09:07:47.643395 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 09:07:47.644557 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 09:07:47.644557 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:07:47.644557 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:07:47.644557 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:07:47.644557 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:07:47.644557 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:07:47.644557 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:07:47.644557 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:07:47.644557 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 14 09:07:47.644557 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:07:47.644557 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:07:47.644557 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:07:47.644557 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:07:47.644557 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:07:47.644557 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:07:47.644557 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:07:47.644557 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:07:47.645893 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 09:07:47.645893 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 09:07:47.645893 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 09:07:47.645893 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 14 09:07:47.645893 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 620/1228] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jul 14 09:07:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 09:07:47.651078 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4426ee47-3f86-4bbd-bf0f-0b09220de8eb None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:07:47.760725 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4426ee47-3f86-4bbd-bf0f-0b09220de8eb None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:07:47.764384 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:07:47.799284 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff767793-b067-45ea-9bc6-3b27877ded63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:07:47.800344 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff767793-b067-45ea-9bc6-3b27877ded63 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:07:47.800727 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff767793-b067-45ea-9bc6-3b27877ded63 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:07:47.801075 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff767793-b067-45ea-9bc6-3b27877ded63 None admin] PATH_INFO: `/v3/domains` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:07:47.801976 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff767793-b067-45ea-9bc6-3b27877ded63 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:07:47.802583 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff767793-b067-45ea-9bc6-3b27877ded63 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:07:47.804600 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff767793-b067-45ea-9bc6-3b27877ded63 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:07:47.816831 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 609/1229] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Jul 14 09:07:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:07:47.822638 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3625d59f-26a0-43ae-a0fd-94b77dbea601 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:07:47.833038 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:07:47.939792 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3625d59f-26a0-43ae-a0fd-94b77dbea601 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:07:47.978986 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1bfb5d-6560-424c-b0de-c27199992c41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:07:47.979861 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1bfb5d-6560-424c-b0de-c27199992c41 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:07:47.980173 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1bfb5d-6560-424c-b0de-c27199992c41 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:07:47.980486 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1bfb5d-6560-424c-b0de-c27199992c41 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:07:47.987930 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1bfb5d-6560-424c-b0de-c27199992c41 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:07:47.989535 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1bfb5d-6560-424c-b0de-c27199992c41 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:07:48.010193 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-9a1bfb5d-6560-424c-b0de-c27199992c41 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 09:07:48.010193 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:07:48.010193 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:07:48.010193 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:07:48.010193 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:07:48.010193 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:07:48.010193 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:07:48.010193 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:07:48.010193 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:07:48.010193 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:07:48.010193 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:07:48.010193 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:07:48.010193 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 09:07:48.010193 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 09:07:48.010193 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 09:07:48.011312 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 09:07:48.011312 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:07:48.011312 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:07:48.011312 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:07:48.011312 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:07:48.011312 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:07:48.011312 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:07:48.011312 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:07:48.011312 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 14 09:07:48.011312 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:07:48.011312 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:07:48.011312 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:07:48.011312 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:07:48.011312 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:07:48.011312 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:07:48.011312 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:07:48.011312 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:07:48.012296 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 09:07:48.012296 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 09:07:48.012296 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 09:07:48.012296 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 09:07:48.012296 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 09:07:48.012296 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 09:07:48.012296 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 09:07:48.012296 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 14 09:07:48.012296 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 621/1230] 10.222.0.26 () {56 vars in 1111 bytes} [Thu Jul 14 09:07:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:07:48.018226 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff767793-b067-45ea-9bc6-3b27877ded63 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:07:48.132063 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff767793-b067-45ea-9bc6-3b27877ded63 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:07:48.170658 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69606e04-72d4-4cba-a28b-fd7fae7d858d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:07:48.171369 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69606e04-72d4-4cba-a28b-fd7fae7d858d None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:07:48.171655 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69606e04-72d4-4cba-a28b-fd7fae7d858d None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:07:48.171933 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69606e04-72d4-4cba-a28b-fd7fae7d858d None admin] PATH_INFO: `/v3/projects` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:07:48.172725 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69606e04-72d4-4cba-a28b-fd7fae7d858d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:07:48.173036 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69606e04-72d4-4cba-a28b-fd7fae7d858d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:07:48.174472 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69606e04-72d4-4cba-a28b-fd7fae7d858d None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:07:48.187293 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 610/1231] 10.222.0.26 () {56 vars in 1148 bytes} [Thu Jul 14 09:07:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:07:48.193370 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1bfb5d-6560-424c-b0de-c27199992c41 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:07:48.265227 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:07:48.306146 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1bfb5d-6560-424c-b0de-c27199992c41 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:07:48.351762 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4cc1f8-4789-4558-b3db-dd7f45e7bed1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:07:48.352960 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4cc1f8-4789-4558-b3db-dd7f45e7bed1 None admin] REQUEST_METHOD: `PUT` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:07:48.353305 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4cc1f8-4789-4558-b3db-dd7f45e7bed1 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:07:48.353556 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4cc1f8-4789-4558-b3db-dd7f45e7bed1 None admin] PATH_INFO: `/v3/projects/33bf210d015847439655b9943d1b0261/users/355684f52e074ad9bfa7fd90bcae6caf/roles/f5c487bc605f45df82ef6367c80fae70` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:07:48.374601 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4cc1f8-4789-4558-b3db-dd7f45e7bed1 None admin] RBAC: Authorizing `identity:create_grant(project_id=33bf210d015847439655b9943d1b0261, user_id=355684f52e074ad9bfa7fd90bcae6caf, role_id=f5c487bc605f45df82ef6367c80fae70)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:07:48.376266 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4cc1f8-4789-4558-b3db-dd7f45e7bed1 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:07:48.391225 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 622/1232] 10.222.0.26 () {58 vars in 1439 bytes} [Thu Jul 14 09:07:48 2022] PUT /identity/v3/projects/33bf210d015847439655b9943d1b0261/users/355684f52e074ad9bfa7fd90bcae6caf/roles/f5c487bc605f45df82ef6367c80fae70 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:07:50.487867 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e389cb-9ad3-4a33-9229-28af3cd734e5 None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:07:50.488440 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e389cb-9ad3-4a33-9229-28af3cd734e5 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:07:50.488440 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e389cb-9ad3-4a33-9229-28af3cd734e5 None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:07:50.489812 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 611/1233] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:07:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:07:50.495672 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb7ce48-8abb-46cf-899e-50e446a339e9 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:07:50.496100 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb7ce48-8abb-46cf-899e-50e446a339e9 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:07:50.496225 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb7ce48-8abb-46cf-899e-50e446a339e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:07:50.514768 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-acb7ce48-8abb-46cf-899e-50e446a339e9 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:07:50.516009 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-acb7ce48-8abb-46cf-899e-50e446a339e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:07:50.585772 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 623/1234] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:07:50 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 09:07:50.611057 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e389cb-9ad3-4a33-9229-28af3cd734e5 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:07:50.638438 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:07:50.717639 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e389cb-9ad3-4a33-9229-28af3cd734e5 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:07:50.753225 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea50f24-5e91-4424-86b1-bbca2e766adf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:07:50.754145 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea50f24-5e91-4424-86b1-bbca2e766adf None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:07:50.754526 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea50f24-5e91-4424-86b1-bbca2e766adf None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:07:50.754901 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea50f24-5e91-4424-86b1-bbca2e766adf None admin] PATH_INFO: `/v3/roles/service` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:07:50.766800 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea50f24-5e91-4424-86b1-bbca2e766adf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 09:07:50.774833 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-0ea50f24-5e91-4424-86b1-bbca2e766adf None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 09:07:50.774833 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:07:50.774833 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:07:50.774833 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:07:50.774833 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:07:50.774833 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:07:50.774833 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:07:50.774833 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:07:50.774833 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:07:50.774833 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:07:50.774833 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:07:50.774833 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:07:50.774833 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 09:07:50.774833 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 09:07:50.774833 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 09:07:50.774833 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 09:07:50.776033 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 09:07:50.776033 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 09:07:50.776033 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 09:07:50.776033 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 09:07:50.776033 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:07:50.776033 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:07:50.776033 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:07:50.776033 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:07:50.776033 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:07:50.776033 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:07:50.776033 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:07:50.776033 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 14 09:07:50.776033 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:07:50.776033 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:07:50.776033 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:07:50.776033 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:07:50.777098 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:07:50.777098 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:07:50.777098 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:07:50.777098 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:07:50.777098 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 09:07:50.777098 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 09:07:50.777098 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 09:07:50.777098 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 09:07:50.777098 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 09:07:50.777098 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 09:07:50.777098 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 09:07:50.777098 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 14 09:07:50.781305 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 612/1235] 10.222.0.26 () {56 vars in 1102 bytes} [Thu Jul 14 09:07:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:07:50.782505 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb7ce48-8abb-46cf-899e-50e446a339e9 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:07:50.832733 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:07:50.888697 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb7ce48-8abb-46cf-899e-50e446a339e9 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:07:50.925724 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e99eb06-0b25-49eb-86e5-532805817a5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:07:50.926436 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e99eb06-0b25-49eb-86e5-532805817a5e None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:07:50.926719 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e99eb06-0b25-49eb-86e5-532805817a5e None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:07:50.927012 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e99eb06-0b25-49eb-86e5-532805817a5e None admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:07:50.927873 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e99eb06-0b25-49eb-86e5-532805817a5e None admin] RBAC: Adding query filter params (name=service) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:07:50.928180 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e99eb06-0b25-49eb-86e5-532805817a5e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:07:50.929707 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e99eb06-0b25-49eb-86e5-532805817a5e None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:07:50.939237 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 624/1236] 10.222.0.26 () {56 vars in 1103 bytes} [Thu Jul 14 09:07:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:07:50.945590 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea50f24-5e91-4424-86b1-bbca2e766adf None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:07:51.042972 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:07:51.060326 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea50f24-5e91-4424-86b1-bbca2e766adf None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:07:51.098327 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64cb108-5a62-4182-854a-31e21e2b255d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:07:51.099128 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64cb108-5a62-4182-854a-31e21e2b255d None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:07:51.099440 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64cb108-5a62-4182-854a-31e21e2b255d None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:07:51.099725 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64cb108-5a62-4182-854a-31e21e2b255d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:07:51.112109 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64cb108-5a62-4182-854a-31e21e2b255d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:07:51.113566 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64cb108-5a62-4182-854a-31e21e2b255d None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:07:51.130842 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-c64cb108-5a62-4182-854a-31e21e2b255d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 09:07:51.130842 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:07:51.130842 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:07:51.130842 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:07:51.130842 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:07:51.130842 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:07:51.130842 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:07:51.130842 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:07:51.130842 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:07:51.130842 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:07:51.130842 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:07:51.130842 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:07:51.130842 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 09:07:51.130842 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 09:07:51.130842 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 09:07:51.132246 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 09:07:51.132246 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:07:51.132246 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:07:51.132246 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:07:51.132246 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:07:51.132246 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:07:51.132246 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:07:51.132246 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:07:51.132246 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 14 09:07:51.132246 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:07:51.132246 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:07:51.132246 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:07:51.132246 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:07:51.132246 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:07:51.132246 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:07:51.132246 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:07:51.132246 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:07:51.133235 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 09:07:51.133235 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 09:07:51.133235 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 09:07:51.133235 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 14 09:07:51.133235 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 613/1237] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jul 14 09:07:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:07:51.137991 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e99eb06-0b25-49eb-86e5-532805817a5e None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:07:51.247812 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e99eb06-0b25-49eb-86e5-532805817a5e None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:07:51.257763 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:07:51.287570 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e4738b-90ed-4d26-ac59-fe54475d8e73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:07:51.288347 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e4738b-90ed-4d26-ac59-fe54475d8e73 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:07:51.288666 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e4738b-90ed-4d26-ac59-fe54475d8e73 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:07:51.288954 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e4738b-90ed-4d26-ac59-fe54475d8e73 None admin] PATH_INFO: `/v3/domains` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:07:51.289741 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e4738b-90ed-4d26-ac59-fe54475d8e73 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:07:51.290050 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e4738b-90ed-4d26-ac59-fe54475d8e73 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:07:51.291469 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e4738b-90ed-4d26-ac59-fe54475d8e73 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:07:51.304896 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 625/1238] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Jul 14 09:07:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:07:51.310911 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64cb108-5a62-4182-854a-31e21e2b255d None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:07:51.429350 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64cb108-5a62-4182-854a-31e21e2b255d None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:07:51.466884 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0a4556-c076-42f8-b77a-3e80120f5ecd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:07:51.467713 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0a4556-c076-42f8-b77a-3e80120f5ecd None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:07:51.468096 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0a4556-c076-42f8-b77a-3e80120f5ecd None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:07:51.468342 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0a4556-c076-42f8-b77a-3e80120f5ecd None admin] PATH_INFO: `/v3/users/placement` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:07:51.471943 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:07:51.478062 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0a4556-c076-42f8-b77a-3e80120f5ecd None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:07:51.480297 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0a4556-c076-42f8-b77a-3e80120f5ecd None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:07:51.496674 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-ed0a4556-c076-42f8-b77a-3e80120f5ecd None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 14 09:07:51.496674 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:07:51.496674 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:07:51.496674 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:07:51.496674 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:07:51.496674 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:07:51.496674 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:07:51.496674 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:07:51.496674 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:07:51.496674 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:07:51.496674 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:07:51.496674 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:07:51.496674 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 09:07:51.496674 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 09:07:51.496674 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 09:07:51.496674 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 09:07:51.498402 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:07:51.498402 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:07:51.498402 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 09:07:51.498402 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 09:07:51.498402 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 09:07:51.498402 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 09:07:51.498402 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:07:51.498402 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:07:51.498402 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:07:51.498402 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:07:51.498402 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:07:51.498402 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 14 09:07:51.498402 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:07:51.498402 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:07:51.498402 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:07:51.498402 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:07:51.498402 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:07:51.498402 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:07:51.499715 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:07:51.499715 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:07:51.499715 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 09:07:51.499715 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 09:07:51.499715 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 09:07:51.499715 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 09:07:51.499715 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 09:07:51.499715 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 09:07:51.499715 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 14 09:07:51.499715 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 14 09:07:51.499715 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 614/1239] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jul 14 09:07:51 2022] GET /identity/v3/users/placement => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:07:51.504044 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e4738b-90ed-4d26-ac59-fe54475d8e73 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:07:51.614091 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e4738b-90ed-4d26-ac59-fe54475d8e73 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:07:51.649545 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5f3b57-8dd3-4281-91be-98d000052dc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:07:51.650233 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5f3b57-8dd3-4281-91be-98d000052dc9 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:07:51.650545 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5f3b57-8dd3-4281-91be-98d000052dc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:07:51.650822 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5f3b57-8dd3-4281-91be-98d000052dc9 None admin] PATH_INFO: `/v3/users` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:07:51.651728 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5f3b57-8dd3-4281-91be-98d000052dc9 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:07:51.652032 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5f3b57-8dd3-4281-91be-98d000052dc9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:07:51.653673 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5f3b57-8dd3-4281-91be-98d000052dc9 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:07:51.671707 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 626/1240] 10.222.0.26 () {56 vars in 1143 bytes} [Thu Jul 14 09:07:51 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:07:51.677522 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0a4556-c076-42f8-b77a-3e80120f5ecd None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:07:51.785694 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0a4556-c076-42f8-b77a-3e80120f5ecd None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:07:51.821235 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4e290b-33ee-496b-8610-75414f608e22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:07:51.821917 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4e290b-33ee-496b-8610-75414f608e22 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:07:51.822174 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4e290b-33ee-496b-8610-75414f608e22 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:07:51.822454 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4e290b-33ee-496b-8610-75414f608e22 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:07:51.892726 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:07:51.901664 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4e290b-33ee-496b-8610-75414f608e22 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:07:51.903601 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4e290b-33ee-496b-8610-75414f608e22 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:07:52.016005 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 615/1241] 10.222.0.26 () {58 vars in 1282 bytes} [Thu Jul 14 09:07:51 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 09:07:52.022708 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5f3b57-8dd3-4281-91be-98d000052dc9 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:07:52.052215 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:07:52.132033 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5f3b57-8dd3-4281-91be-98d000052dc9 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:07:52.171412 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d916247e-bbde-42cb-b32b-d61cb138d5c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:07:52.172286 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d916247e-bbde-42cb-b32b-d61cb138d5c9 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:07:52.172685 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d916247e-bbde-42cb-b32b-d61cb138d5c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:07:52.173073 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d916247e-bbde-42cb-b32b-d61cb138d5c9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:07:52.183535 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d916247e-bbde-42cb-b32b-d61cb138d5c9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:07:52.185044 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d916247e-bbde-42cb-b32b-d61cb138d5c9 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:07:52.203537 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-d916247e-bbde-42cb-b32b-d61cb138d5c9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 09:07:52.203537 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:07:52.203537 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:07:52.203537 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:07:52.203537 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:07:52.203537 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:07:52.203537 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:07:52.203537 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:07:52.203537 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:07:52.203537 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:07:52.203537 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:07:52.203537 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:07:52.203537 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 09:07:52.203537 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 09:07:52.203537 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 09:07:52.205860 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 09:07:52.205860 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:07:52.205860 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:07:52.205860 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:07:52.205860 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:07:52.205860 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:07:52.205860 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:07:52.205860 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:07:52.205860 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 14 09:07:52.205860 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:07:52.205860 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:07:52.205860 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:07:52.205860 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:07:52.205860 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:07:52.205860 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:07:52.205860 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:07:52.205860 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:07:52.207010 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 09:07:52.207010 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 09:07:52.207010 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 09:07:52.207010 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 14 09:07:52.207010 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 627/1242] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jul 14 09:07:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 09:07:52.210738 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4e290b-33ee-496b-8610-75414f608e22 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:07:52.316677 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:07:52.323208 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4e290b-33ee-496b-8610-75414f608e22 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:07:52.364838 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b29492-8684-48f4-8d54-a6f4d2f6ed70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:07:52.365735 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b29492-8684-48f4-8d54-a6f4d2f6ed70 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:07:52.366030 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b29492-8684-48f4-8d54-a6f4d2f6ed70 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:07:52.366565 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b29492-8684-48f4-8d54-a6f4d2f6ed70 None admin] PATH_INFO: `/v3/domains` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:07:52.367549 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b29492-8684-48f4-8d54-a6f4d2f6ed70 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:07:52.367977 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b29492-8684-48f4-8d54-a6f4d2f6ed70 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:07:52.370031 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b29492-8684-48f4-8d54-a6f4d2f6ed70 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:07:52.385632 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 616/1243] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Jul 14 09:07:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:07:52.391466 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d916247e-bbde-42cb-b32b-d61cb138d5c9 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:07:52.475805 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:07:52.500834 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d916247e-bbde-42cb-b32b-d61cb138d5c9 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:07:52.539934 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25eb7a07-549b-4c37-a050-47cc3061b32f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:07:52.540767 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25eb7a07-549b-4c37-a050-47cc3061b32f None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:07:52.541046 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25eb7a07-549b-4c37-a050-47cc3061b32f None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:07:52.541315 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25eb7a07-549b-4c37-a050-47cc3061b32f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:07:52.548437 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25eb7a07-549b-4c37-a050-47cc3061b32f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:07:52.550015 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25eb7a07-549b-4c37-a050-47cc3061b32f None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:07:52.565390 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-25eb7a07-549b-4c37-a050-47cc3061b32f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 09:07:52.565390 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:07:52.565390 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:07:52.565390 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:07:52.565390 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:07:52.565390 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:07:52.565390 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:07:52.565390 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:07:52.565390 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:07:52.565390 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:07:52.565390 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:07:52.565390 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:07:52.565390 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 09:07:52.565390 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 09:07:52.565390 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 09:07:52.566486 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 09:07:52.566486 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:07:52.566486 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:07:52.566486 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:07:52.566486 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:07:52.566486 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:07:52.566486 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:07:52.566486 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:07:52.566486 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 14 09:07:52.566486 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:07:52.566486 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:07:52.566486 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:07:52.566486 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:07:52.566486 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:07:52.566486 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:07:52.566486 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:07:52.566486 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:07:52.567630 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 09:07:52.567630 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 09:07:52.567630 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 09:07:52.567630 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 09:07:52.567630 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 09:07:52.567630 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 09:07:52.567630 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 09:07:52.567630 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 14 09:07:52.567630 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 628/1244] 10.222.0.26 () {56 vars in 1111 bytes} [Thu Jul 14 09:07:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:07:52.573401 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b29492-8684-48f4-8d54-a6f4d2f6ed70 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:07:52.683268 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b29492-8684-48f4-8d54-a6f4d2f6ed70 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:07:52.724389 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26b4600-0621-416d-8e61-6024f9ee3569 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:07:52.725143 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26b4600-0621-416d-8e61-6024f9ee3569 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:07:52.725531 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26b4600-0621-416d-8e61-6024f9ee3569 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:07:52.725858 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26b4600-0621-416d-8e61-6024f9ee3569 None admin] PATH_INFO: `/v3/projects` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:07:52.726819 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26b4600-0621-416d-8e61-6024f9ee3569 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:07:52.727306 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26b4600-0621-416d-8e61-6024f9ee3569 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:07:52.729034 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26b4600-0621-416d-8e61-6024f9ee3569 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:07:52.745331 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 617/1245] 10.222.0.26 () {56 vars in 1148 bytes} [Thu Jul 14 09:07:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:07:52.751573 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25eb7a07-549b-4c37-a050-47cc3061b32f None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:07:52.856272 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25eb7a07-549b-4c37-a050-47cc3061b32f None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:07:52.891065 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f60542-609b-415d-ae4c-b21c42be7beb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:07:52.891761 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f60542-609b-415d-ae4c-b21c42be7beb None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:07:52.892045 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f60542-609b-415d-ae4c-b21c42be7beb None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:07:52.892324 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f60542-609b-415d-ae4c-b21c42be7beb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:07:52.893176 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f60542-609b-415d-ae4c-b21c42be7beb None admin] RBAC: Adding query filter params (role.id=f5c487bc605f45df82ef6367c80fae70, scope.project.id=33bf210d015847439655b9943d1b0261, user.id=355684f52e074ad9bfa7fd90bcae6caf) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:07:52.893484 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f60542-609b-415d-ae4c-b21c42be7beb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:07:52.894915 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f60542-609b-415d-ae4c-b21c42be7beb None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:07:52.901075 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 629/1246] 10.222.0.26 () {56 vars in 1374 bytes} [Thu Jul 14 09:07:52 2022] GET /identity/v3/role_assignments?user.id=355684f52e074ad9bfa7fd90bcae6caf&scope.project.id=33bf210d015847439655b9943d1b0261&role.id=f5c487bc605f45df82ef6367c80fae70 => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:07:55.069563 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef01006-f191-4284-934e-b65733689364 None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:07:55.070126 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef01006-f191-4284-934e-b65733689364 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:07:55.070221 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef01006-f191-4284-934e-b65733689364 None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:07:55.071749 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 618/1247] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:07:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:07:55.077078 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c339b6-5186-48b4-bf61-72b7a564cb85 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:07:55.077540 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c339b6-5186-48b4-bf61-72b7a564cb85 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:07:55.077634 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c339b6-5186-48b4-bf61-72b7a564cb85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:07:55.083215 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:07:55.096585 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-c0c339b6-5186-48b4-bf61-72b7a564cb85 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:07:55.097857 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-c0c339b6-5186-48b4-bf61-72b7a564cb85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:07:55.174168 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 630/1248] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:07:55 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 09:07:55.199056 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef01006-f191-4284-934e-b65733689364 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:07:55.221529 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:07:55.337361 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef01006-f191-4284-934e-b65733689364 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:07:55.382479 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4c15b1-a708-4dd0-8dd9-c4781e8c9be6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:07:55.383503 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4c15b1-a708-4dd0-8dd9-c4781e8c9be6 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:07:55.383939 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4c15b1-a708-4dd0-8dd9-c4781e8c9be6 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:07:55.384301 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4c15b1-a708-4dd0-8dd9-c4781e8c9be6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:07:55.397795 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4c15b1-a708-4dd0-8dd9-c4781e8c9be6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 09:07:55.406488 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-2f4c15b1-a708-4dd0-8dd9-c4781e8c9be6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 09:07:55.406488 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:07:55.406488 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:07:55.406488 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:07:55.406488 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:07:55.406488 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:07:55.406488 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:07:55.406488 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:07:55.406488 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:07:55.406488 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:07:55.406488 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:07:55.406488 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:07:55.406488 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 09:07:55.406488 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 09:07:55.406488 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 09:07:55.406488 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 09:07:55.407650 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 09:07:55.407650 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 09:07:55.407650 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 09:07:55.407650 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 09:07:55.407650 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:07:55.407650 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:07:55.407650 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:07:55.407650 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:07:55.407650 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:07:55.407650 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:07:55.407650 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:07:55.407650 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 14 09:07:55.407650 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:07:55.407650 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:07:55.407650 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:07:55.407650 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:07:55.408515 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:07:55.408515 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:07:55.408515 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:07:55.408515 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:07:55.408515 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 09:07:55.408515 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 09:07:55.408515 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 09:07:55.408515 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 09:07:55.408515 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 09:07:55.408515 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 09:07:55.408515 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 09:07:55.408515 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 14 09:07:55.408515 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 619/1249] 10.222.0.26 () {56 vars in 1096 bytes} [Thu Jul 14 09:07:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 09:07:55.416504 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c339b6-5186-48b4-bf61-72b7a564cb85 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:07:55.523803 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c339b6-5186-48b4-bf61-72b7a564cb85 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:07:55.557967 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:07:55.564109 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d5487c-adaf-4589-8343-775e32003f17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:07:55.564860 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d5487c-adaf-4589-8343-775e32003f17 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:07:55.565106 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d5487c-adaf-4589-8343-775e32003f17 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:07:55.565382 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d5487c-adaf-4589-8343-775e32003f17 None admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:07:55.566137 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d5487c-adaf-4589-8343-775e32003f17 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:07:55.566444 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d5487c-adaf-4589-8343-775e32003f17 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:07:55.567894 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d5487c-adaf-4589-8343-775e32003f17 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:07:55.577252 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 631/1250] 10.222.0.26 () {56 vars in 1099 bytes} [Thu Jul 14 09:07:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:07:55.583689 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4c15b1-a708-4dd0-8dd9-c4781e8c9be6 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:07:55.671247 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:07:55.695569 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4c15b1-a708-4dd0-8dd9-c4781e8c9be6 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:07:55.734472 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4403b9-7c15-41a1-9d65-6aabdaa54a1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:07:55.735523 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4403b9-7c15-41a1-9d65-6aabdaa54a1e None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:07:55.735805 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4403b9-7c15-41a1-9d65-6aabdaa54a1e None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:07:55.736078 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4403b9-7c15-41a1-9d65-6aabdaa54a1e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:07:55.749009 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4403b9-7c15-41a1-9d65-6aabdaa54a1e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:07:55.750463 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4403b9-7c15-41a1-9d65-6aabdaa54a1e None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:07:55.771044 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-0a4403b9-7c15-41a1-9d65-6aabdaa54a1e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 09:07:55.771044 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:07:55.771044 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:07:55.771044 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:07:55.771044 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:07:55.771044 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:07:55.771044 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:07:55.771044 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:07:55.771044 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:07:55.771044 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:07:55.771044 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:07:55.771044 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:07:55.771044 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 09:07:55.771044 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 09:07:55.771044 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 09:07:55.772275 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 09:07:55.772275 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:07:55.772275 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:07:55.772275 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:07:55.772275 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:07:55.772275 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:07:55.772275 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:07:55.772275 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:07:55.772275 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 14 09:07:55.772275 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:07:55.772275 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:07:55.772275 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:07:55.772275 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:07:55.772275 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:07:55.772275 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:07:55.772275 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:07:55.772275 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:07:55.773627 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 09:07:55.773627 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 09:07:55.773627 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 09:07:55.773627 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 14 09:07:55.773627 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 620/1251] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jul 14 09:07:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:07:55.778835 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d5487c-adaf-4589-8343-775e32003f17 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:07:55.896399 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d5487c-adaf-4589-8343-775e32003f17 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:07:55.933549 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fea5858-9e18-47e7-9603-b131e2d11ff0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:07:55.934266 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fea5858-9e18-47e7-9603-b131e2d11ff0 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:07:55.934557 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fea5858-9e18-47e7-9603-b131e2d11ff0 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:07:55.934887 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fea5858-9e18-47e7-9603-b131e2d11ff0 None admin] PATH_INFO: `/v3/domains` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:07:55.935726 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fea5858-9e18-47e7-9603-b131e2d11ff0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:07:55.936075 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fea5858-9e18-47e7-9603-b131e2d11ff0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:07:55.937638 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fea5858-9e18-47e7-9603-b131e2d11ff0 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:07:55.950705 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 632/1252] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Jul 14 09:07:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:07:55.956779 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4403b9-7c15-41a1-9d65-6aabdaa54a1e None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:07:56.074434 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4403b9-7c15-41a1-9d65-6aabdaa54a1e None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:07:56.102786 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:07:56.112656 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddc66b9-ed3f-481c-b66d-7e8b67132946 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:07:56.113473 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddc66b9-ed3f-481c-b66d-7e8b67132946 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:07:56.113761 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddc66b9-ed3f-481c-b66d-7e8b67132946 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:07:56.114038 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddc66b9-ed3f-481c-b66d-7e8b67132946 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:07:56.122645 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddc66b9-ed3f-481c-b66d-7e8b67132946 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:07:56.124239 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddc66b9-ed3f-481c-b66d-7e8b67132946 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:07:56.140037 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-8ddc66b9-ed3f-481c-b66d-7e8b67132946 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 14 09:07:56.140037 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:07:56.140037 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:07:56.140037 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:07:56.140037 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:07:56.140037 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:07:56.140037 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:07:56.140037 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:07:56.140037 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:07:56.140037 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:07:56.140037 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:07:56.140037 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:07:56.140037 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 09:07:56.140037 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 09:07:56.140037 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 09:07:56.140037 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 09:07:56.141506 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:07:56.141506 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:07:56.141506 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 09:07:56.141506 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 09:07:56.141506 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 09:07:56.141506 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 09:07:56.141506 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:07:56.141506 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:07:56.141506 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:07:56.141506 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:07:56.141506 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:07:56.141506 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 14 09:07:56.141506 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:07:56.141506 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:07:56.141506 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:07:56.141506 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:07:56.141506 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:07:56.141506 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:07:56.142511 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:07:56.142511 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:07:56.142511 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 09:07:56.142511 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 09:07:56.142511 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 09:07:56.142511 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 09:07:56.142511 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 09:07:56.142511 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 09:07:56.142511 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 14 09:07:56.142511 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 14 09:07:56.142511 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 621/1253] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jul 14 09:07:55 2022] GET /identity/v3/users/placement => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:07:56.147667 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fea5858-9e18-47e7-9603-b131e2d11ff0 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:07:56.210752 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:07:56.260877 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fea5858-9e18-47e7-9603-b131e2d11ff0 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:07:56.300159 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d3354a-6b2f-4b5f-94e2-ec882f92e59d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:07:56.300860 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d3354a-6b2f-4b5f-94e2-ec882f92e59d None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:07:56.301211 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d3354a-6b2f-4b5f-94e2-ec882f92e59d None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:07:56.301474 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d3354a-6b2f-4b5f-94e2-ec882f92e59d None admin] PATH_INFO: `/v3/users` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:07:56.302378 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d3354a-6b2f-4b5f-94e2-ec882f92e59d None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:07:56.302708 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d3354a-6b2f-4b5f-94e2-ec882f92e59d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:07:56.304191 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d3354a-6b2f-4b5f-94e2-ec882f92e59d None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:07:56.322522 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 633/1254] 10.222.0.26 () {56 vars in 1143 bytes} [Thu Jul 14 09:07:56 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 09:07:56.328852 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddc66b9-ed3f-481c-b66d-7e8b67132946 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:07:56.444058 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddc66b9-ed3f-481c-b66d-7e8b67132946 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:07:56.491603 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf96b8c-f95a-4d96-91d5-84ea955adab4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:07:56.492654 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf96b8c-f95a-4d96-91d5-84ea955adab4 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:07:56.493192 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf96b8c-f95a-4d96-91d5-84ea955adab4 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:07:56.493638 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf96b8c-f95a-4d96-91d5-84ea955adab4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:07:56.563034 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:07:56.580207 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf96b8c-f95a-4d96-91d5-84ea955adab4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:07:56.581910 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf96b8c-f95a-4d96-91d5-84ea955adab4 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:07:56.699485 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 622/1255] 10.222.0.26 () {58 vars in 1282 bytes} [Thu Jul 14 09:07:56 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 09:07:56.708207 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d3354a-6b2f-4b5f-94e2-ec882f92e59d None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:07:56.829338 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d3354a-6b2f-4b5f-94e2-ec882f92e59d None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:07:56.847127 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:07:56.875569 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fb8761-b9fa-4e31-9896-c6cd24ee60da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:07:56.876502 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fb8761-b9fa-4e31-9896-c6cd24ee60da None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:07:56.876874 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fb8761-b9fa-4e31-9896-c6cd24ee60da None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:07:56.877151 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fb8761-b9fa-4e31-9896-c6cd24ee60da None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:07:56.889599 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fb8761-b9fa-4e31-9896-c6cd24ee60da None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:07:56.891080 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fb8761-b9fa-4e31-9896-c6cd24ee60da None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:07:56.916965 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-d8fb8761-b9fa-4e31-9896-c6cd24ee60da None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 09:07:56.916965 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:07:56.916965 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:07:56.916965 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:07:56.916965 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:07:56.916965 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:07:56.916965 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:07:56.916965 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:07:56.916965 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:07:56.916965 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:07:56.916965 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:07:56.916965 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:07:56.916965 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 09:07:56.916965 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 09:07:56.916965 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 09:07:56.918049 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 09:07:56.918049 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:07:56.918049 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:07:56.918049 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:07:56.918049 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:07:56.918049 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:07:56.918049 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:07:56.918049 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:07:56.918049 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 14 09:07:56.918049 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:07:56.918049 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:07:56.918049 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:07:56.918049 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:07:56.918049 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:07:56.918049 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:07:56.918049 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:07:56.918049 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:07:56.924059 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 09:07:56.924059 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 09:07:56.924059 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 09:07:56.924059 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 14 09:07:56.924059 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 634/1256] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jul 14 09:07:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:07:56.924403 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf96b8c-f95a-4d96-91d5-84ea955adab4 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:07:57.019339 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:07:57.032338 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf96b8c-f95a-4d96-91d5-84ea955adab4 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:07:57.066890 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3dc1aa-2119-486e-8b7a-83800e403305 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:07:57.067596 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3dc1aa-2119-486e-8b7a-83800e403305 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:07:57.067883 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3dc1aa-2119-486e-8b7a-83800e403305 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:07:57.068209 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3dc1aa-2119-486e-8b7a-83800e403305 None admin] PATH_INFO: `/v3/domains` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:07:57.068880 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3dc1aa-2119-486e-8b7a-83800e403305 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:07:57.069189 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3dc1aa-2119-486e-8b7a-83800e403305 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:07:57.070575 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3dc1aa-2119-486e-8b7a-83800e403305 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:07:57.082366 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 623/1257] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Jul 14 09:07:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:07:57.088172 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fb8761-b9fa-4e31-9896-c6cd24ee60da None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:07:57.191016 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fb8761-b9fa-4e31-9896-c6cd24ee60da None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:07:57.224254 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda6ed69-45e8-4f99-b19b-b3742e037503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:07:57.225074 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda6ed69-45e8-4f99-b19b-b3742e037503 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:07:57.225339 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda6ed69-45e8-4f99-b19b-b3742e037503 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:07:57.225607 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda6ed69-45e8-4f99-b19b-b3742e037503 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:07:57.232544 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda6ed69-45e8-4f99-b19b-b3742e037503 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:07:57.233994 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda6ed69-45e8-4f99-b19b-b3742e037503 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:07:57.248596 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-dda6ed69-45e8-4f99-b19b-b3742e037503 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 09:07:57.248596 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:07:57.248596 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:07:57.248596 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:07:57.248596 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:07:57.248596 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:07:57.248596 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:07:57.248596 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:07:57.248596 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:07:57.248596 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:07:57.248596 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:07:57.248596 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:07:57.248596 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 09:07:57.248596 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 09:07:57.248596 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 09:07:57.249830 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 09:07:57.249830 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:07:57.249830 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:07:57.249830 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:07:57.249830 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:07:57.249830 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:07:57.249830 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:07:57.249830 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:07:57.249830 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 14 09:07:57.249830 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:07:57.249830 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:07:57.249830 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:07:57.249830 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:07:57.249830 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:07:57.249830 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:07:57.249830 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:07:57.249830 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:07:57.255876 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 09:07:57.255876 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 09:07:57.255876 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 09:07:57.255876 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 09:07:57.255876 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 09:07:57.255876 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 09:07:57.255876 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 09:07:57.255876 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 14 09:07:57.255876 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 635/1258] 10.222.0.26 () {56 vars in 1111 bytes} [Thu Jul 14 09:07:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:07:57.256467 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3dc1aa-2119-486e-8b7a-83800e403305 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:07:57.369480 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3dc1aa-2119-486e-8b7a-83800e403305 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:07:57.407130 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b05391-f8fc-4b8b-a22b-9293f8110376 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:07:57.407878 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b05391-f8fc-4b8b-a22b-9293f8110376 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:07:57.408159 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b05391-f8fc-4b8b-a22b-9293f8110376 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:07:57.408437 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b05391-f8fc-4b8b-a22b-9293f8110376 None admin] PATH_INFO: `/v3/projects` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:07:57.409226 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b05391-f8fc-4b8b-a22b-9293f8110376 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:07:57.409527 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b05391-f8fc-4b8b-a22b-9293f8110376 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:07:57.410968 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b05391-f8fc-4b8b-a22b-9293f8110376 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:07:57.423620 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 624/1259] 10.222.0.26 () {56 vars in 1148 bytes} [Thu Jul 14 09:07:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:07:57.430392 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda6ed69-45e8-4f99-b19b-b3742e037503 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:07:57.459899 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:07:57.542294 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda6ed69-45e8-4f99-b19b-b3742e037503 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:07:57.577782 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0bb1ed-da2b-42a6-bacb-652c63f52c16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:07:57.578480 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0bb1ed-da2b-42a6-bacb-652c63f52c16 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:07:57.578766 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0bb1ed-da2b-42a6-bacb-652c63f52c16 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:07:57.579037 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0bb1ed-da2b-42a6-bacb-652c63f52c16 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:07:57.579907 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0bb1ed-da2b-42a6-bacb-652c63f52c16 None admin] RBAC: Adding query filter params (role.id=7b549d58d1724395b343609d895c80d0, scope.project.id=33bf210d015847439655b9943d1b0261, user.id=355684f52e074ad9bfa7fd90bcae6caf) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:07:57.580217 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0bb1ed-da2b-42a6-bacb-652c63f52c16 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:07:57.581668 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0bb1ed-da2b-42a6-bacb-652c63f52c16 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:07:57.587435 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 636/1260] 10.222.0.26 () {56 vars in 1374 bytes} [Thu Jul 14 09:07:57 2022] GET /identity/v3/role_assignments?user.id=355684f52e074ad9bfa7fd90bcae6caf&scope.project.id=33bf210d015847439655b9943d1b0261&role.id=7b549d58d1724395b343609d895c80d0 => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:07:59.577907 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b0b326-f386-44aa-a075-a2db5fb692fe None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:07:59.578362 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b0b326-f386-44aa-a075-a2db5fb692fe None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:07:59.578477 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b0b326-f386-44aa-a075-a2db5fb692fe None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:07:59.579889 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 625/1261] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:07:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:07:59.585259 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dd9fd8-ce96-46e8-b6db-58691e564047 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:07:59.585708 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dd9fd8-ce96-46e8-b6db-58691e564047 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:07:59.585800 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dd9fd8-ce96-46e8-b6db-58691e564047 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:07:59.604321 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-17dd9fd8-ce96-46e8-b6db-58691e564047 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:07:59.605490 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-17dd9fd8-ce96-46e8-b6db-58691e564047 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:07:59.676860 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 637/1262] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:07:59 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 09:07:59.702595 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b0b326-f386-44aa-a075-a2db5fb692fe None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:07:59.718587 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:07:59.818248 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b0b326-f386-44aa-a075-a2db5fb692fe None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:07:59.853629 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b1c25a-9931-414d-9bc6-9684fb086cef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:07:59.854422 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b1c25a-9931-414d-9bc6-9684fb086cef None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:07:59.854708 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b1c25a-9931-414d-9bc6-9684fb086cef None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:07:59.854980 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b1c25a-9931-414d-9bc6-9684fb086cef None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:07:59.865866 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b1c25a-9931-414d-9bc6-9684fb086cef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 09:07:59.873870 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-98b1c25a-9931-414d-9bc6-9684fb086cef None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 09:07:59.873870 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:07:59.873870 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:07:59.873870 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:07:59.873870 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:07:59.873870 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:07:59.873870 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:07:59.873870 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:07:59.873870 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:07:59.873870 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:07:59.873870 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:07:59.873870 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:07:59.873870 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 09:07:59.873870 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 09:07:59.873870 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 09:07:59.873870 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 09:07:59.879296 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 09:07:59.879296 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 09:07:59.879296 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 09:07:59.879296 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 09:07:59.879296 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:07:59.879296 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:07:59.879296 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:07:59.879296 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:07:59.879296 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:07:59.879296 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:07:59.879296 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:07:59.879296 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 14 09:07:59.879296 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:07:59.879296 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:07:59.879296 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:07:59.879296 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:07:59.880617 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:07:59.880617 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:07:59.880617 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:07:59.880617 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:07:59.880617 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 09:07:59.880617 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 09:07:59.880617 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 09:07:59.880617 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 09:07:59.880617 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 09:07:59.880617 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 09:07:59.880617 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 09:07:59.880617 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 14 09:07:59.880617 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 626/1263] 10.222.0.26 () {56 vars in 1096 bytes} [Thu Jul 14 09:07:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:07:59.882190 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17dd9fd8-ce96-46e8-b6db-58691e564047 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:07:59.901629 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:07:59.985969 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17dd9fd8-ce96-46e8-b6db-58691e564047 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:08:00.021628 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce767917-a27f-4d73-a557-c5505ad75f1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:08:00.022280 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce767917-a27f-4d73-a557-c5505ad75f1f None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:08:00.022560 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce767917-a27f-4d73-a557-c5505ad75f1f None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:08:00.022834 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce767917-a27f-4d73-a557-c5505ad75f1f None admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:08:00.023615 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce767917-a27f-4d73-a557-c5505ad75f1f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:08:00.023920 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce767917-a27f-4d73-a557-c5505ad75f1f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:08:00.025319 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce767917-a27f-4d73-a557-c5505ad75f1f None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:08:00.033545 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 638/1264] 10.222.0.26 () {56 vars in 1099 bytes} [Thu Jul 14 09:07:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:08:00.040096 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b1c25a-9931-414d-9bc6-9684fb086cef None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:08:00.130874 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:08:00.174330 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b1c25a-9931-414d-9bc6-9684fb086cef None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:08:00.210672 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d465ebe7-985b-486c-b7fa-b088bfa0197c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:08:00.211496 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d465ebe7-985b-486c-b7fa-b088bfa0197c None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:08:00.211783 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d465ebe7-985b-486c-b7fa-b088bfa0197c None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:08:00.212060 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d465ebe7-985b-486c-b7fa-b088bfa0197c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:08:00.223486 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d465ebe7-985b-486c-b7fa-b088bfa0197c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:08:00.224940 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d465ebe7-985b-486c-b7fa-b088bfa0197c None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:08:00.242798 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-d465ebe7-985b-486c-b7fa-b088bfa0197c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 09:08:00.242798 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:08:00.242798 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:08:00.242798 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:08:00.242798 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:08:00.242798 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:08:00.242798 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:08:00.242798 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:08:00.242798 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:08:00.242798 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:08:00.242798 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:08:00.242798 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:08:00.242798 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 09:08:00.242798 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 09:08:00.242798 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 09:08:00.243924 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 09:08:00.243924 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:08:00.243924 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:08:00.243924 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:08:00.243924 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:08:00.243924 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:08:00.243924 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:08:00.243924 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:08:00.243924 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 14 09:08:00.243924 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:08:00.243924 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:08:00.243924 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:08:00.243924 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:08:00.243924 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:08:00.243924 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:08:00.243924 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:08:00.243924 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:08:00.244987 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 09:08:00.244987 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 09:08:00.244987 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 09:08:00.244987 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 14 09:08:00.244987 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 627/1265] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jul 14 09:08:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:08:00.250387 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce767917-a27f-4d73-a557-c5505ad75f1f None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:08:00.337889 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:08:00.362011 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce767917-a27f-4d73-a557-c5505ad75f1f None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:08:00.402133 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7c3931-1ae7-411c-a42e-cea34d4832e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:08:00.402736 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7c3931-1ae7-411c-a42e-cea34d4832e5 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:08:00.403018 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7c3931-1ae7-411c-a42e-cea34d4832e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:08:00.403328 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7c3931-1ae7-411c-a42e-cea34d4832e5 None admin] PATH_INFO: `/v3/domains` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:08:00.404064 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7c3931-1ae7-411c-a42e-cea34d4832e5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:08:00.404371 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7c3931-1ae7-411c-a42e-cea34d4832e5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:08:00.405747 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7c3931-1ae7-411c-a42e-cea34d4832e5 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:08:00.418215 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 639/1266] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Jul 14 09:08:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:08:00.424400 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d465ebe7-985b-486c-b7fa-b088bfa0197c None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:08:00.549787 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d465ebe7-985b-486c-b7fa-b088bfa0197c None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:08:00.572269 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:08:00.587764 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c8bcfa-259b-40bd-95e0-c10639e92189 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:08:00.588565 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c8bcfa-259b-40bd-95e0-c10639e92189 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:08:00.588856 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c8bcfa-259b-40bd-95e0-c10639e92189 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:08:00.589135 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c8bcfa-259b-40bd-95e0-c10639e92189 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:08:00.596671 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c8bcfa-259b-40bd-95e0-c10639e92189 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:08:00.598138 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c8bcfa-259b-40bd-95e0-c10639e92189 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:08:00.612373 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-85c8bcfa-259b-40bd-95e0-c10639e92189 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 14 09:08:00.612373 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:08:00.612373 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:08:00.612373 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:08:00.612373 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:08:00.612373 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:08:00.612373 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:08:00.612373 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:08:00.612373 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:08:00.612373 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:08:00.612373 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:08:00.612373 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:08:00.612373 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 09:08:00.612373 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 09:08:00.612373 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 09:08:00.612373 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 09:08:00.613441 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:08:00.613441 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:08:00.613441 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 09:08:00.613441 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 09:08:00.613441 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 09:08:00.613441 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 09:08:00.613441 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:08:00.613441 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:08:00.613441 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:08:00.613441 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:08:00.613441 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:08:00.613441 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 14 09:08:00.613441 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:08:00.613441 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:08:00.613441 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:08:00.613441 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:08:00.613441 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:08:00.613441 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:08:00.614761 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:08:00.614761 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:08:00.614761 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 09:08:00.614761 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 09:08:00.614761 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 09:08:00.614761 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 09:08:00.614761 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 09:08:00.614761 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 09:08:00.614761 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 14 09:08:00.614761 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 14 09:08:00.614761 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 628/1267] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jul 14 09:08:00 2022] GET /identity/v3/users/placement => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:08:00.619586 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7c3931-1ae7-411c-a42e-cea34d4832e5 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:08:00.723613 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7c3931-1ae7-411c-a42e-cea34d4832e5 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:08:00.759770 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70412486-4b4c-469b-b753-ac1279b2b114 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:08:00.760717 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70412486-4b4c-469b-b753-ac1279b2b114 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:08:00.761079 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70412486-4b4c-469b-b753-ac1279b2b114 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:08:00.761427 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70412486-4b4c-469b-b753-ac1279b2b114 None admin] PATH_INFO: `/v3/users` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:08:00.762757 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70412486-4b4c-469b-b753-ac1279b2b114 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:08:00.763184 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70412486-4b4c-469b-b753-ac1279b2b114 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:08:00.765429 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70412486-4b4c-469b-b753-ac1279b2b114 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:08:00.779734 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:08:00.792404 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 640/1268] 10.222.0.26 () {56 vars in 1143 bytes} [Thu Jul 14 09:08:00 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:08:00.798182 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c8bcfa-259b-40bd-95e0-c10639e92189 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:08:00.913299 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c8bcfa-259b-40bd-95e0-c10639e92189 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:08:00.952466 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6e528d-6c3b-4f10-9b52-b070c3cbb34c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:08:00.953240 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6e528d-6c3b-4f10-9b52-b070c3cbb34c None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:08:00.953525 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6e528d-6c3b-4f10-9b52-b070c3cbb34c None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:08:00.953796 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6e528d-6c3b-4f10-9b52-b070c3cbb34c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:08:00.965415 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6e528d-6c3b-4f10-9b52-b070c3cbb34c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:08:00.966848 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6e528d-6c3b-4f10-9b52-b070c3cbb34c None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:08:00.984424 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-0e6e528d-6c3b-4f10-9b52-b070c3cbb34c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 09:08:00.984424 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:08:00.984424 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:08:00.984424 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:08:00.984424 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:08:00.984424 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:08:00.984424 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:08:00.984424 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:08:00.984424 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:08:00.984424 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:08:00.984424 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:08:00.984424 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:08:00.984424 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 09:08:00.984424 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 09:08:00.984424 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 09:08:00.985428 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 09:08:00.985428 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:08:00.985428 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:08:00.985428 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:08:00.985428 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:08:00.985428 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:08:00.985428 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:08:00.985428 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:08:00.985428 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 14 09:08:00.985428 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:08:00.985428 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:08:00.985428 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:08:00.985428 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:08:00.985428 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:08:00.985428 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:08:00.985428 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:08:00.985428 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:08:00.986482 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 09:08:00.986482 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 09:08:00.986482 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 09:08:00.986482 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 14 09:08:00.986482 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 629/1269] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jul 14 09:08:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:08:00.991428 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70412486-4b4c-469b-b753-ac1279b2b114 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:08:01.100368 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70412486-4b4c-469b-b753-ac1279b2b114 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:08:01.135149 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be864f9-540d-4e03-bf81-cfe58952964b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:08:01.135818 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be864f9-540d-4e03-bf81-cfe58952964b None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:08:01.136092 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be864f9-540d-4e03-bf81-cfe58952964b None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:08:01.136361 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be864f9-540d-4e03-bf81-cfe58952964b None admin] PATH_INFO: `/v3/domains` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:08:01.137076 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be864f9-540d-4e03-bf81-cfe58952964b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:08:01.137379 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be864f9-540d-4e03-bf81-cfe58952964b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:08:01.138728 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be864f9-540d-4e03-bf81-cfe58952964b None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:08:01.150036 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 641/1270] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Jul 14 09:08:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:08:01.156163 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6e528d-6c3b-4f10-9b52-b070c3cbb34c None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:08:01.202315 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:08:01.273334 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6e528d-6c3b-4f10-9b52-b070c3cbb34c None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:08:01.307549 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d06dd77-f8d3-4ca5-814c-24fbac82f0ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:08:01.308324 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d06dd77-f8d3-4ca5-814c-24fbac82f0ec None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:08:01.308606 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d06dd77-f8d3-4ca5-814c-24fbac82f0ec None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:08:01.308882 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d06dd77-f8d3-4ca5-814c-24fbac82f0ec None admin] PATH_INFO: `/v3/projects/service` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:08:01.316437 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d06dd77-f8d3-4ca5-814c-24fbac82f0ec None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:08:01.317934 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d06dd77-f8d3-4ca5-814c-24fbac82f0ec None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:08:01.337391 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-2d06dd77-f8d3-4ca5-814c-24fbac82f0ec None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 09:08:01.337391 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:08:01.337391 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:08:01.337391 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:08:01.337391 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:08:01.337391 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:08:01.337391 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:08:01.337391 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:08:01.337391 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:08:01.337391 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:08:01.337391 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:08:01.337391 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:08:01.337391 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 09:08:01.337391 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 09:08:01.337391 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 09:08:01.338722 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 09:08:01.338722 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:08:01.338722 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:08:01.338722 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:08:01.338722 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:08:01.338722 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:08:01.338722 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:08:01.338722 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:08:01.338722 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 14 09:08:01.338722 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:08:01.338722 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:08:01.338722 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:08:01.338722 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:08:01.338722 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:08:01.338722 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:08:01.338722 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:08:01.338722 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:08:01.340040 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 09:08:01.340040 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 09:08:01.340040 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 09:08:01.340040 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 09:08:01.340040 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 09:08:01.340040 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 09:08:01.340040 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 09:08:01.340040 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 14 09:08:01.340040 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 630/1271] 10.222.0.26 () {56 vars in 1111 bytes} [Thu Jul 14 09:08:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:08:01.346669 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be864f9-540d-4e03-bf81-cfe58952964b None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:08:01.415387 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:08:01.461095 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be864f9-540d-4e03-bf81-cfe58952964b None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:08:01.502244 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e14285d-891d-40a1-94b2-616c79a736cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:08:01.502899 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e14285d-891d-40a1-94b2-616c79a736cd None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:08:01.503178 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e14285d-891d-40a1-94b2-616c79a736cd None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:08:01.503489 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e14285d-891d-40a1-94b2-616c79a736cd None admin] PATH_INFO: `/v3/projects` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:08:01.504274 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e14285d-891d-40a1-94b2-616c79a736cd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:08:01.504585 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e14285d-891d-40a1-94b2-616c79a736cd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:08:01.506010 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e14285d-891d-40a1-94b2-616c79a736cd None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:08:01.518103 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 642/1272] 10.222.0.26 () {56 vars in 1148 bytes} [Thu Jul 14 09:08:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:08:01.524440 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d06dd77-f8d3-4ca5-814c-24fbac82f0ec None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:08:01.623139 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:08:01.634654 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d06dd77-f8d3-4ca5-814c-24fbac82f0ec None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:08:01.671735 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30277b2d-893b-42db-93cc-4f65234e5156 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:08:01.672630 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30277b2d-893b-42db-93cc-4f65234e5156 None admin] REQUEST_METHOD: `PUT` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:08:01.672914 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30277b2d-893b-42db-93cc-4f65234e5156 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:08:01.673191 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30277b2d-893b-42db-93cc-4f65234e5156 None admin] PATH_INFO: `/v3/projects/33bf210d015847439655b9943d1b0261/users/355684f52e074ad9bfa7fd90bcae6caf/roles/7b549d58d1724395b343609d895c80d0` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:08:01.675455 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30277b2d-893b-42db-93cc-4f65234e5156 None admin] RBAC: Authorizing `identity:create_grant(project_id=33bf210d015847439655b9943d1b0261, user_id=355684f52e074ad9bfa7fd90bcae6caf, role_id=7b549d58d1724395b343609d895c80d0)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:08:01.676905 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30277b2d-893b-42db-93cc-4f65234e5156 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:08:01.693255 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 631/1273] 10.222.0.26 () {58 vars in 1439 bytes} [Thu Jul 14 09:08:01 2022] PUT /identity/v3/projects/33bf210d015847439655b9943d1b0261/users/355684f52e074ad9bfa7fd90bcae6caf/roles/7b549d58d1724395b343609d895c80d0 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:08:03.688098 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e710eae7-e25b-4882-9f36-8534349bd1c6 None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:08:03.688892 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e710eae7-e25b-4882-9f36-8534349bd1c6 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:08:03.688892 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e710eae7-e25b-4882-9f36-8534349bd1c6 None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:08:03.691124 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 643/1274] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:08:03 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:08:03.697556 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d07e5cb-a159-4705-9e7c-2b2206514872 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:08:03.698026 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d07e5cb-a159-4705-9e7c-2b2206514872 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:08:03.698139 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d07e5cb-a159-4705-9e7c-2b2206514872 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:08:03.717686 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-3d07e5cb-a159-4705-9e7c-2b2206514872 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:08:03.718877 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-3d07e5cb-a159-4705-9e7c-2b2206514872 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:08:03.797071 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 632/1275] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:08:03 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 09:08:03.820486 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e710eae7-e25b-4882-9f36-8534349bd1c6 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:08:03.924330 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e710eae7-e25b-4882-9f36-8534349bd1c6 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:08:03.941643 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:08:03.961281 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8858b9-8263-463e-bf89-90aab3945097 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:08:03.962067 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8858b9-8263-463e-bf89-90aab3945097 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:08:03.962347 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8858b9-8263-463e-bf89-90aab3945097 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:08:03.962626 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8858b9-8263-463e-bf89-90aab3945097 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:08:03.973495 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8858b9-8263-463e-bf89-90aab3945097 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 09:08:03.981177 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-df8858b9-8263-463e-bf89-90aab3945097 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 09:08:03.981177 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:08:03.981177 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:08:03.981177 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:08:03.981177 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:08:03.981177 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:08:03.981177 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:08:03.981177 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:08:03.981177 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:08:03.981177 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:08:03.981177 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:08:03.981177 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:08:03.981177 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 09:08:03.981177 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 09:08:03.981177 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 09:08:03.981177 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 09:08:03.982249 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 09:08:03.982249 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 09:08:03.982249 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 09:08:03.982249 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 09:08:03.982249 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:08:03.982249 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:08:03.982249 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:08:03.982249 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:08:03.982249 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:08:03.982249 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:08:03.982249 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:08:03.982249 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 14 09:08:03.982249 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:08:03.982249 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:08:03.982249 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:08:03.982249 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:08:03.983122 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:08:03.983122 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:08:03.983122 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:08:03.983122 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:08:03.983122 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 09:08:03.983122 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 09:08:03.983122 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 09:08:03.983122 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 09:08:03.983122 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 09:08:03.983122 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 09:08:03.983122 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 09:08:03.983122 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 14 09:08:03.983122 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 644/1276] 10.222.0.26 () {56 vars in 1096 bytes} [Thu Jul 14 09:08:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:08:03.988512 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d07e5cb-a159-4705-9e7c-2b2206514872 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:08:04.071600 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:08:04.090871 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d07e5cb-a159-4705-9e7c-2b2206514872 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:08:04.126504 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed979a2-7c20-4c00-8ad2-ee342e85e6e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:08:04.127159 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed979a2-7c20-4c00-8ad2-ee342e85e6e5 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:08:04.127477 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed979a2-7c20-4c00-8ad2-ee342e85e6e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:08:04.127746 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed979a2-7c20-4c00-8ad2-ee342e85e6e5 None admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:08:04.128495 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed979a2-7c20-4c00-8ad2-ee342e85e6e5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:08:04.128808 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed979a2-7c20-4c00-8ad2-ee342e85e6e5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:08:04.130164 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed979a2-7c20-4c00-8ad2-ee342e85e6e5 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:08:04.138323 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 633/1277] 10.222.0.26 () {56 vars in 1099 bytes} [Thu Jul 14 09:08:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:08:04.144833 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8858b9-8263-463e-bf89-90aab3945097 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:08:04.251617 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8858b9-8263-463e-bf89-90aab3945097 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:08:04.288950 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8597c56-4538-4b23-a765-206cd57c27ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:08:04.289737 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8597c56-4538-4b23-a765-206cd57c27ad None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:08:04.290019 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8597c56-4538-4b23-a765-206cd57c27ad None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:08:04.290293 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8597c56-4538-4b23-a765-206cd57c27ad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:08:04.301046 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8597c56-4538-4b23-a765-206cd57c27ad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:08:04.302495 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8597c56-4538-4b23-a765-206cd57c27ad None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:08:04.319802 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-c8597c56-4538-4b23-a765-206cd57c27ad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 09:08:04.319802 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:08:04.319802 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:08:04.319802 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:08:04.319802 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:08:04.319802 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:08:04.319802 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:08:04.319802 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:08:04.319802 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:08:04.319802 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:08:04.319802 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:08:04.319802 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:08:04.319802 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 09:08:04.319802 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 09:08:04.319802 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 09:08:04.320965 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 09:08:04.320965 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:08:04.320965 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:08:04.320965 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:08:04.320965 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:08:04.320965 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:08:04.320965 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:08:04.320965 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:08:04.320965 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 14 09:08:04.320965 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:08:04.320965 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:08:04.320965 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:08:04.320965 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:08:04.320965 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:08:04.320965 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:08:04.320965 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:08:04.320965 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:08:04.322645 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 09:08:04.322645 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 09:08:04.322645 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 09:08:04.322645 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 14 09:08:04.322645 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 645/1278] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jul 14 09:08:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:08:04.327841 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed979a2-7c20-4c00-8ad2-ee342e85e6e5 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:08:04.430169 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed979a2-7c20-4c00-8ad2-ee342e85e6e5 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:08:04.465136 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1e9216-35a7-4b95-945b-4589bbdb04d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:08:04.465810 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1e9216-35a7-4b95-945b-4589bbdb04d5 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:08:04.466092 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1e9216-35a7-4b95-945b-4589bbdb04d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:08:04.466373 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1e9216-35a7-4b95-945b-4589bbdb04d5 None admin] PATH_INFO: `/v3/domains` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:08:04.467092 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1e9216-35a7-4b95-945b-4589bbdb04d5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:08:04.467429 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1e9216-35a7-4b95-945b-4589bbdb04d5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:08:04.468807 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1e9216-35a7-4b95-945b-4589bbdb04d5 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:08:04.480903 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 634/1279] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Jul 14 09:08:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:08:04.486815 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8597c56-4538-4b23-a765-206cd57c27ad None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:08:04.521915 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:08:04.596043 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8597c56-4538-4b23-a765-206cd57c27ad None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:08:04.631208 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eec3631-d2d0-4a87-93d3-f68854784544 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:08:04.632011 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eec3631-d2d0-4a87-93d3-f68854784544 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:08:04.632294 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eec3631-d2d0-4a87-93d3-f68854784544 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:08:04.632568 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eec3631-d2d0-4a87-93d3-f68854784544 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:08:04.641210 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eec3631-d2d0-4a87-93d3-f68854784544 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:08:04.642684 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eec3631-d2d0-4a87-93d3-f68854784544 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:08:04.657050 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-0eec3631-d2d0-4a87-93d3-f68854784544 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 14 09:08:04.657050 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:08:04.657050 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:08:04.657050 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:08:04.657050 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:08:04.657050 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:08:04.657050 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:08:04.657050 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:08:04.657050 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:08:04.657050 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:08:04.657050 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:08:04.657050 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:08:04.657050 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 09:08:04.657050 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 09:08:04.657050 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 09:08:04.657050 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 09:08:04.658048 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:08:04.658048 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:08:04.658048 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 09:08:04.658048 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 09:08:04.658048 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 09:08:04.658048 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 09:08:04.658048 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:08:04.658048 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:08:04.658048 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:08:04.658048 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:08:04.658048 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:08:04.658048 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 14 09:08:04.658048 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:08:04.658048 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:08:04.658048 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:08:04.658048 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:08:04.658048 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:08:04.658048 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:08:04.658999 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:08:04.658999 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:08:04.658999 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 09:08:04.658999 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 09:08:04.658999 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 09:08:04.658999 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 09:08:04.658999 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 09:08:04.658999 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 09:08:04.658999 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 14 09:08:04.658999 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 14 09:08:04.658999 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 646/1280] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jul 14 09:08:04 2022] GET /identity/v3/users/placement => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:08:04.666600 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1e9216-35a7-4b95-945b-4589bbdb04d5 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:08:04.670890 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:08:04.772181 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1e9216-35a7-4b95-945b-4589bbdb04d5 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:08:04.806950 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a817cd04-b017-485d-ba64-6e5028bac8dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:08:04.807653 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a817cd04-b017-485d-ba64-6e5028bac8dd None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:08:04.807932 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a817cd04-b017-485d-ba64-6e5028bac8dd None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:08:04.808207 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a817cd04-b017-485d-ba64-6e5028bac8dd None admin] PATH_INFO: `/v3/users` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:08:04.809039 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a817cd04-b017-485d-ba64-6e5028bac8dd None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:08:04.809342 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a817cd04-b017-485d-ba64-6e5028bac8dd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:08:04.810765 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a817cd04-b017-485d-ba64-6e5028bac8dd None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:08:04.827881 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 635/1281] 10.222.0.26 () {56 vars in 1143 bytes} [Thu Jul 14 09:08:04 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:08:04.834366 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eec3631-d2d0-4a87-93d3-f68854784544 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:08:04.934401 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:08:04.950083 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eec3631-d2d0-4a87-93d3-f68854784544 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:08:04.991668 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edabca3-568f-49d2-be9b-7ec2aedba4ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:08:04.992311 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edabca3-568f-49d2-be9b-7ec2aedba4ee None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:08:04.992598 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edabca3-568f-49d2-be9b-7ec2aedba4ee None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:08:04.992872 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edabca3-568f-49d2-be9b-7ec2aedba4ee None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:08:05.066329 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edabca3-568f-49d2-be9b-7ec2aedba4ee None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:08:05.067835 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edabca3-568f-49d2-be9b-7ec2aedba4ee None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:08:05.162039 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:08:05.172877 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 647/1282] 10.222.0.26 () {58 vars in 1282 bytes} [Thu Jul 14 09:08:04 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 09:08:05.181928 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a817cd04-b017-485d-ba64-6e5028bac8dd None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:08:05.250735 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:08:05.290654 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a817cd04-b017-485d-ba64-6e5028bac8dd None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:08:05.328202 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e247b6c-831d-4ded-8cfb-7eceeb0c4567 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:08:05.329014 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e247b6c-831d-4ded-8cfb-7eceeb0c4567 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:08:05.329291 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e247b6c-831d-4ded-8cfb-7eceeb0c4567 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:08:05.329717 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e247b6c-831d-4ded-8cfb-7eceeb0c4567 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:08:05.341587 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e247b6c-831d-4ded-8cfb-7eceeb0c4567 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:08:05.343043 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e247b6c-831d-4ded-8cfb-7eceeb0c4567 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:08:05.361135 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-1e247b6c-831d-4ded-8cfb-7eceeb0c4567 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 09:08:05.361135 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:08:05.361135 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:08:05.361135 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:08:05.361135 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:08:05.361135 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:08:05.361135 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:08:05.361135 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:08:05.361135 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:08:05.361135 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:08:05.361135 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:08:05.361135 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:08:05.361135 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 09:08:05.361135 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 09:08:05.361135 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 09:08:05.362137 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 09:08:05.362137 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:08:05.362137 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:08:05.362137 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:08:05.362137 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:08:05.362137 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:08:05.362137 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:08:05.362137 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:08:05.362137 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 14 09:08:05.362137 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:08:05.362137 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:08:05.362137 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:08:05.362137 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:08:05.362137 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:08:05.362137 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:08:05.362137 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:08:05.362137 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:08:05.363107 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 09:08:05.363107 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 09:08:05.363107 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 09:08:05.363107 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 14 09:08:05.363107 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 636/1283] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jul 14 09:08:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:08:05.368283 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edabca3-568f-49d2-be9b-7ec2aedba4ee None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:08:05.471949 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edabca3-568f-49d2-be9b-7ec2aedba4ee None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:08:05.507521 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b67649-f41b-42f9-9752-ea293eb60e2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:08:05.508113 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b67649-f41b-42f9-9752-ea293eb60e2b None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:08:05.508400 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b67649-f41b-42f9-9752-ea293eb60e2b None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:08:05.508673 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b67649-f41b-42f9-9752-ea293eb60e2b None admin] PATH_INFO: `/v3/domains` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:08:05.509397 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b67649-f41b-42f9-9752-ea293eb60e2b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:08:05.509696 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b67649-f41b-42f9-9752-ea293eb60e2b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:08:05.511099 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b67649-f41b-42f9-9752-ea293eb60e2b None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:08:05.522541 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 648/1284] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Jul 14 09:08:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:08:05.529497 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e247b6c-831d-4ded-8cfb-7eceeb0c4567 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:08:05.633954 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e247b6c-831d-4ded-8cfb-7eceeb0c4567 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:08:05.648313 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:08:05.670462 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7d5db8-56a2-49a9-bcd1-ca280b5a85e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:08:05.671281 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7d5db8-56a2-49a9-bcd1-ca280b5a85e9 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:08:05.671559 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7d5db8-56a2-49a9-bcd1-ca280b5a85e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:08:05.671834 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7d5db8-56a2-49a9-bcd1-ca280b5a85e9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:08:05.678665 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7d5db8-56a2-49a9-bcd1-ca280b5a85e9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:08:05.680142 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7d5db8-56a2-49a9-bcd1-ca280b5a85e9 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:08:05.693876 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-fd7d5db8-56a2-49a9-bcd1-ca280b5a85e9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 09:08:05.693876 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:08:05.693876 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:08:05.693876 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:08:05.693876 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:08:05.693876 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:08:05.693876 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:08:05.693876 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:08:05.693876 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:08:05.693876 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:08:05.693876 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:08:05.693876 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:08:05.693876 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 09:08:05.693876 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 09:08:05.693876 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 09:08:05.694969 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 09:08:05.694969 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:08:05.694969 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:08:05.694969 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:08:05.694969 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:08:05.694969 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:08:05.694969 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:08:05.694969 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:08:05.694969 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 14 09:08:05.694969 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:08:05.694969 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:08:05.694969 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:08:05.694969 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:08:05.694969 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:08:05.694969 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:08:05.694969 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:08:05.694969 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:08:05.696119 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 09:08:05.696119 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 09:08:05.696119 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 09:08:05.696119 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 09:08:05.696119 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 09:08:05.696119 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 09:08:05.696119 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 09:08:05.696119 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 14 09:08:05.696119 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 637/1285] 10.222.0.26 () {56 vars in 1111 bytes} [Thu Jul 14 09:08:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:08:05.701330 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b67649-f41b-42f9-9752-ea293eb60e2b None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:08:05.754905 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:08:05.809181 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b67649-f41b-42f9-9752-ea293eb60e2b None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:08:05.843644 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a20c86-95d0-4c6b-a1ae-966b0366cedc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:08:05.844250 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a20c86-95d0-4c6b-a1ae-966b0366cedc None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:08:05.844567 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a20c86-95d0-4c6b-a1ae-966b0366cedc None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:08:05.844844 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a20c86-95d0-4c6b-a1ae-966b0366cedc None admin] PATH_INFO: `/v3/projects` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:08:05.845640 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a20c86-95d0-4c6b-a1ae-966b0366cedc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:08:05.845952 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a20c86-95d0-4c6b-a1ae-966b0366cedc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:08:05.847405 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a20c86-95d0-4c6b-a1ae-966b0366cedc None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:08:05.859781 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 649/1286] 10.222.0.26 () {56 vars in 1148 bytes} [Thu Jul 14 09:08:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:08:05.866297 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7d5db8-56a2-49a9-bcd1-ca280b5a85e9 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:08:05.973690 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7d5db8-56a2-49a9-bcd1-ca280b5a85e9 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:08:06.013695 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab67fc69-4de7-4d24-b074-e7d40b089e3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:08:06.014327 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab67fc69-4de7-4d24-b074-e7d40b089e3f None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:08:06.014609 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab67fc69-4de7-4d24-b074-e7d40b089e3f None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:08:06.014880 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab67fc69-4de7-4d24-b074-e7d40b089e3f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:08:06.015734 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab67fc69-4de7-4d24-b074-e7d40b089e3f None admin] RBAC: Adding query filter params (role.id=7b549d58d1724395b343609d895c80d0, scope.project.id=33bf210d015847439655b9943d1b0261, user.id=355684f52e074ad9bfa7fd90bcae6caf) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:08:06.016040 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab67fc69-4de7-4d24-b074-e7d40b089e3f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:08:06.017462 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab67fc69-4de7-4d24-b074-e7d40b089e3f None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:08:06.023422 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 638/1287] 10.222.0.26 () {56 vars in 1374 bytes} [Thu Jul 14 09:08:05 2022] GET /identity/v3/role_assignments?user.id=355684f52e074ad9bfa7fd90bcae6caf&scope.project.id=33bf210d015847439655b9943d1b0261&role.id=7b549d58d1724395b343609d895c80d0 => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:08:08.035265 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69688c78-dc49-4648-a5c7-4392b7830d53 None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:08:08.036103 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69688c78-dc49-4648-a5c7-4392b7830d53 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:08:08.036248 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69688c78-dc49-4648-a5c7-4392b7830d53 None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:08:08.038405 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 650/1288] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:08:08 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:08:08.046327 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6a9378-2111-4157-8cba-3f2e94d95226 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:08:08.046792 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6a9378-2111-4157-8cba-3f2e94d95226 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:08:08.046887 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6a9378-2111-4157-8cba-3f2e94d95226 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:08:08.064913 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-8c6a9378-2111-4157-8cba-3f2e94d95226 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:08:08.066071 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-8c6a9378-2111-4157-8cba-3f2e94d95226 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:08:08.096503 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:08:08.139092 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 639/1289] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:08:08 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 09:08:08.161876 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69688c78-dc49-4648-a5c7-4392b7830d53 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:08:08.308356 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69688c78-dc49-4648-a5c7-4392b7830d53 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:08:08.324763 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:08:08.351897 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4468db-aa26-426a-a1a1-a3eb4157586d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:08:08.352774 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4468db-aa26-426a-a1a1-a3eb4157586d None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:08:08.353091 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4468db-aa26-426a-a1a1-a3eb4157586d None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:08:08.353411 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4468db-aa26-426a-a1a1-a3eb4157586d None admin] PATH_INFO: `/v3/services/placement` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:08:08.354155 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4468db-aa26-426a-a1a1-a3eb4157586d None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:08:08.355794 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4468db-aa26-426a-a1a1-a3eb4157586d None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:08:08.369916 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-9c4468db-aa26-426a-a1a1-a3eb4157586d None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 14 09:08:08.369916 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:08:08.369916 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 09:08:08.369916 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 09:08:08.369916 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 09:08:08.369916 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 09:08:08.369916 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 09:08:08.369916 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 09:08:08.369916 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 14 09:08:08.369916 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 14 09:08:08.369916 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 09:08:08.369916 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 14 09:08:08.369916 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:08:08.369916 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:08:08.369916 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:08:08.369916 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:08:08.369916 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:08:08.371244 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:08:08.371244 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:08:08.371244 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:08:08.371244 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:08:08.371244 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:08:08.371244 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:08:08.371244 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 09:08:08.371244 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 09:08:08.371244 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 09:08:08.371244 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 09:08:08.371244 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:08:08.371244 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:08:08.371244 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:08:08.371244 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:08:08.371244 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:08:08.371244 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:08:08.372685 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:08:08.372685 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 14 09:08:08.372685 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:08:08.372685 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:08:08.372685 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:08:08.372685 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:08:08.372685 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:08:08.372685 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:08:08.372685 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:08:08.372685 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:08:08.372685 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 09:08:08.372685 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 09:08:08.372685 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 14 09:08:08.372685 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 14 09:08:08.372685 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 651/1290] 10.222.0.26 () {56 vars in 1117 bytes} [Thu Jul 14 09:08:08 2022] GET /identity/v3/services/placement => generated 90 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:08:08.378245 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6a9378-2111-4157-8cba-3f2e94d95226 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:08:08.488191 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6a9378-2111-4157-8cba-3f2e94d95226 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:08:08.527891 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1c1922-fc62-4dbb-8105-3fbee5a4d0b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:08:08.528554 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1c1922-fc62-4dbb-8105-3fbee5a4d0b2 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:08:08.528836 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1c1922-fc62-4dbb-8105-3fbee5a4d0b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:08:08.529144 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1c1922-fc62-4dbb-8105-3fbee5a4d0b2 None admin] PATH_INFO: `/v3/services` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:08:08.529853 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1c1922-fc62-4dbb-8105-3fbee5a4d0b2 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:08:08.530165 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1c1922-fc62-4dbb-8105-3fbee5a4d0b2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:08:08.531575 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1c1922-fc62-4dbb-8105-3fbee5a4d0b2 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:08:08.538827 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 640/1291] 10.222.0.26 () {56 vars in 1116 bytes} [Thu Jul 14 09:08:08 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:08:08.544913 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4468db-aa26-426a-a1a1-a3eb4157586d None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:08:08.648871 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4468db-aa26-426a-a1a1-a3eb4157586d None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:08:08.683886 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ae5c07-ba24-4fb8-95dd-646f6eea8845 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:08:08.684540 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ae5c07-ba24-4fb8-95dd-646f6eea8845 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:08:08.684819 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ae5c07-ba24-4fb8-95dd-646f6eea8845 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:08:08.685095 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ae5c07-ba24-4fb8-95dd-646f6eea8845 None admin] PATH_INFO: `/v3/services` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:08:08.685807 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ae5c07-ba24-4fb8-95dd-646f6eea8845 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:08:08.686117 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ae5c07-ba24-4fb8-95dd-646f6eea8845 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:08:08.687532 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ae5c07-ba24-4fb8-95dd-646f6eea8845 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:08:08.692186 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 652/1292] 10.222.0.26 () {56 vars in 1116 bytes} [Thu Jul 14 09:08:08 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:08:10.670678 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98371100-c7a0-4c34-99af-cb7b251363a1 None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:08:10.671382 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98371100-c7a0-4c34-99af-cb7b251363a1 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:08:10.671382 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98371100-c7a0-4c34-99af-cb7b251363a1 None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:08:10.673484 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 641/1293] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:08:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:08:10.682405 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3d2d25-1617-4c1d-9910-4b63c3a8be98 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:08:10.683226 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3d2d25-1617-4c1d-9910-4b63c3a8be98 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:08:10.683341 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3d2d25-1617-4c1d-9910-4b63c3a8be98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:08:10.703228 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-7a3d2d25-1617-4c1d-9910-4b63c3a8be98 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:08:10.704406 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-7a3d2d25-1617-4c1d-9910-4b63c3a8be98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:08:10.739775 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:08:10.782068 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 653/1294] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:08:10 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 09:08:10.805084 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98371100-c7a0-4c34-99af-cb7b251363a1 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:08:10.837761 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:08:10.924371 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98371100-c7a0-4c34-99af-cb7b251363a1 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:08:10.958786 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c57eb2f-9320-4766-aa74-dbaf7239873c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:08:10.959453 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c57eb2f-9320-4766-aa74-dbaf7239873c None admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:08:10.959735 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c57eb2f-9320-4766-aa74-dbaf7239873c None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:08:10.960013 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c57eb2f-9320-4766-aa74-dbaf7239873c None admin] PATH_INFO: `/v3/services` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:08:10.960733 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c57eb2f-9320-4766-aa74-dbaf7239873c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:08:10.962106 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c57eb2f-9320-4766-aa74-dbaf7239873c None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:08:10.983775 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 642/1295] 10.222.0.26 () {60 vars in 1141 bytes} [Thu Jul 14 09:08:10 2022] POST /identity/v3/services => generated 246 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 09:08:13.029837 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b74c05-47ba-4908-8dcf-5731e53b0a57 None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:08:13.030409 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b74c05-47ba-4908-8dcf-5731e53b0a57 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:08:13.030409 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b74c05-47ba-4908-8dcf-5731e53b0a57 None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:08:13.031825 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 654/1296] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:08:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:08:13.037962 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f7e795-abae-4cc8-a80f-0fcf15a3dc06 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:08:13.038422 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f7e795-abae-4cc8-a80f-0fcf15a3dc06 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:08:13.038541 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f7e795-abae-4cc8-a80f-0fcf15a3dc06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:08:13.063259 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-b6f7e795-abae-4cc8-a80f-0fcf15a3dc06 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:08:13.064460 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-b6f7e795-abae-4cc8-a80f-0fcf15a3dc06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:08:13.142433 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 643/1297] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:08:13 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 09:08:13.165133 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b74c05-47ba-4908-8dcf-5731e53b0a57 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:08:13.294356 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b74c05-47ba-4908-8dcf-5731e53b0a57 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:08:13.334911 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993cb107-19d1-40cf-9e9a-82b6e019dd5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:08:13.336255 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993cb107-19d1-40cf-9e9a-82b6e019dd5f None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:08:13.336491 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993cb107-19d1-40cf-9e9a-82b6e019dd5f None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:08:13.336909 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993cb107-19d1-40cf-9e9a-82b6e019dd5f None admin] PATH_INFO: `/v3/services/placement` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:08:13.337963 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993cb107-19d1-40cf-9e9a-82b6e019dd5f None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:08:13.339797 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993cb107-19d1-40cf-9e9a-82b6e019dd5f None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:08:13.356423 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-993cb107-19d1-40cf-9e9a-82b6e019dd5f None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 14 09:08:13.356423 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:08:13.356423 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 09:08:13.356423 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 09:08:13.356423 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 09:08:13.356423 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 09:08:13.356423 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 09:08:13.356423 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 09:08:13.356423 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 14 09:08:13.356423 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 14 09:08:13.356423 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 09:08:13.356423 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 14 09:08:13.356423 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:08:13.356423 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:08:13.356423 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:08:13.356423 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:08:13.356423 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:08:13.357739 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:08:13.357739 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:08:13.357739 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:08:13.357739 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:08:13.357739 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:08:13.357739 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:08:13.357739 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 09:08:13.357739 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 09:08:13.357739 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 09:08:13.357739 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 09:08:13.357739 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:08:13.357739 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:08:13.357739 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:08:13.357739 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:08:13.357739 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:08:13.357739 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:08:13.359066 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:08:13.359066 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 14 09:08:13.359066 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:08:13.359066 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:08:13.359066 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:08:13.359066 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:08:13.359066 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:08:13.359066 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:08:13.359066 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:08:13.359066 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:08:13.359066 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 09:08:13.359066 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 09:08:13.359066 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 14 09:08:13.359066 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 14 09:08:13.359066 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 655/1298] 10.222.0.26 () {56 vars in 1117 bytes} [Thu Jul 14 09:08:13 2022] GET /identity/v3/services/placement => generated 90 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:08:13.364440 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f7e795-abae-4cc8-a80f-0fcf15a3dc06 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:08:13.389644 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:08:13.475355 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f7e795-abae-4cc8-a80f-0fcf15a3dc06 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:08:13.510842 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2daf2762-0fbf-4d9a-9e7b-d6e0db53d9a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:08:13.511570 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2daf2762-0fbf-4d9a-9e7b-d6e0db53d9a9 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:08:13.511849 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2daf2762-0fbf-4d9a-9e7b-d6e0db53d9a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:08:13.512121 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2daf2762-0fbf-4d9a-9e7b-d6e0db53d9a9 None admin] PATH_INFO: `/v3/services` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:08:13.512833 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2daf2762-0fbf-4d9a-9e7b-d6e0db53d9a9 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:08:13.513136 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2daf2762-0fbf-4d9a-9e7b-d6e0db53d9a9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:08:13.514503 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2daf2762-0fbf-4d9a-9e7b-d6e0db53d9a9 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:08:13.519561 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 644/1299] 10.222.0.26 () {56 vars in 1116 bytes} [Thu Jul 14 09:08:13 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:08:13.526183 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993cb107-19d1-40cf-9e9a-82b6e019dd5f None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:08:13.560123 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:08:13.628783 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993cb107-19d1-40cf-9e9a-82b6e019dd5f None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:08:13.663142 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034a5edb-e2d1-42c6-8911-4e0201178a72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:08:13.663832 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034a5edb-e2d1-42c6-8911-4e0201178a72 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:08:13.664114 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034a5edb-e2d1-42c6-8911-4e0201178a72 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:08:13.664387 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034a5edb-e2d1-42c6-8911-4e0201178a72 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:08:13.665179 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034a5edb-e2d1-42c6-8911-4e0201178a72 None admin] RBAC: Adding query filter params (interface=public, service_id=2227caabb29a4668ab5b236e9696d4f1, region_id=RegionOne) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:08:13.665490 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034a5edb-e2d1-42c6-8911-4e0201178a72 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:08:13.666857 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034a5edb-e2d1-42c6-8911-4e0201178a72 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:08:13.672250 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 656/1300] 10.222.0.26 () {56 vars in 1251 bytes} [Thu Jul 14 09:08:13 2022] GET /identity/v3/endpoints?service_id=2227caabb29a4668ab5b236e9696d4f1&interface=public®ion_id=RegionOne => generated 194 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:08:13.678184 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2daf2762-0fbf-4d9a-9e7b-d6e0db53d9a9 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:08:13.758292 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:08:13.779226 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2daf2762-0fbf-4d9a-9e7b-d6e0db53d9a9 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:08:13.813922 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c442ac9-5654-4897-bdb6-c0b9486035d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:08:13.814501 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c442ac9-5654-4897-bdb6-c0b9486035d2 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:08:13.814780 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c442ac9-5654-4897-bdb6-c0b9486035d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:08:13.815052 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c442ac9-5654-4897-bdb6-c0b9486035d2 None admin] PATH_INFO: `/v3/services` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:08:13.815756 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c442ac9-5654-4897-bdb6-c0b9486035d2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:08:13.817113 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c442ac9-5654-4897-bdb6-c0b9486035d2 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:08:13.824013 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 645/1301] 10.222.0.26 () {56 vars in 1087 bytes} [Thu Jul 14 09:08:13 2022] GET /identity/v3/services => generated 2191 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 09:08:15.813965 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0705f546-388b-499d-8778-a6004d8c09fb None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:08:15.814496 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0705f546-388b-499d-8778-a6004d8c09fb None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:08:15.814496 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0705f546-388b-499d-8778-a6004d8c09fb None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:08:15.817022 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 657/1302] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:08:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:08:15.823699 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727fb7ca-e55f-4a8f-9089-4cac0ef56958 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:08:15.824159 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727fb7ca-e55f-4a8f-9089-4cac0ef56958 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:08:15.824252 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727fb7ca-e55f-4a8f-9089-4cac0ef56958 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:08:15.843840 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-727fb7ca-e55f-4a8f-9089-4cac0ef56958 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:08:15.845833 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-727fb7ca-e55f-4a8f-9089-4cac0ef56958 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:08:15.932374 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 646/1303] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:08:15 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 09:08:15.968117 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0705f546-388b-499d-8778-a6004d8c09fb None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:08:16.071177 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:08:16.079354 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0705f546-388b-499d-8778-a6004d8c09fb None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:08:16.115798 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21af4103-396d-4be3-a20f-a02f7b7f7e02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:08:16.116588 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21af4103-396d-4be3-a20f-a02f7b7f7e02 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:08:16.116872 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21af4103-396d-4be3-a20f-a02f7b7f7e02 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:08:16.117142 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21af4103-396d-4be3-a20f-a02f7b7f7e02 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:08:16.117798 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21af4103-396d-4be3-a20f-a02f7b7f7e02 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:08:16.119172 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21af4103-396d-4be3-a20f-a02f7b7f7e02 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:08:16.131749 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-21af4103-396d-4be3-a20f-a02f7b7f7e02 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 14 09:08:16.131749 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:08:16.131749 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 09:08:16.131749 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 09:08:16.131749 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 09:08:16.131749 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 09:08:16.131749 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 09:08:16.131749 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 09:08:16.131749 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 14 09:08:16.131749 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 14 09:08:16.131749 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 09:08:16.131749 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 14 09:08:16.131749 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:08:16.131749 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:08:16.131749 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:08:16.131749 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:08:16.131749 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:08:16.132887 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:08:16.132887 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:08:16.132887 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:08:16.132887 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:08:16.132887 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:08:16.132887 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:08:16.132887 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 09:08:16.132887 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 09:08:16.132887 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 09:08:16.132887 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 09:08:16.132887 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:08:16.132887 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:08:16.132887 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:08:16.132887 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:08:16.132887 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:08:16.132887 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:08:16.133906 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:08:16.133906 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 14 09:08:16.133906 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:08:16.133906 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:08:16.133906 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:08:16.133906 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:08:16.133906 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:08:16.133906 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:08:16.133906 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:08:16.133906 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:08:16.133906 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 09:08:16.133906 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 09:08:16.133906 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 14 09:08:16.133906 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 14 09:08:16.133906 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 658/1304] 10.222.0.26 () {56 vars in 1117 bytes} [Thu Jul 14 09:08:15 2022] GET /identity/v3/services/placement => generated 90 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:08:16.140876 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727fb7ca-e55f-4a8f-9089-4cac0ef56958 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:08:16.219587 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:08:16.247124 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727fb7ca-e55f-4a8f-9089-4cac0ef56958 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:08:16.282609 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce5be86-f2b8-4601-ad42-c4e8a8830dcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:08:16.283343 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce5be86-f2b8-4601-ad42-c4e8a8830dcb None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:08:16.283625 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce5be86-f2b8-4601-ad42-c4e8a8830dcb None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:08:16.283904 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce5be86-f2b8-4601-ad42-c4e8a8830dcb None admin] PATH_INFO: `/v3/services` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:08:16.284596 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce5be86-f2b8-4601-ad42-c4e8a8830dcb None admin] RBAC: Adding query filter params (name=placement) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:08:16.284897 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce5be86-f2b8-4601-ad42-c4e8a8830dcb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:08:16.286260 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce5be86-f2b8-4601-ad42-c4e8a8830dcb None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:08:16.291511 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 647/1305] 10.222.0.26 () {56 vars in 1116 bytes} [Thu Jul 14 09:08:16 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:08:16.297550 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21af4103-396d-4be3-a20f-a02f7b7f7e02 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:08:16.412552 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21af4103-396d-4be3-a20f-a02f7b7f7e02 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:08:16.448009 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4b1e73-9167-4984-8513-22daddff2ef8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:08:16.448760 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4b1e73-9167-4984-8513-22daddff2ef8 None admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:08:16.449043 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4b1e73-9167-4984-8513-22daddff2ef8 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:08:16.449320 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4b1e73-9167-4984-8513-22daddff2ef8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:08:16.450067 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4b1e73-9167-4984-8513-22daddff2ef8 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:08:16.451478 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4b1e73-9167-4984-8513-22daddff2ef8 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:08:16.471969 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 659/1306] 10.222.0.26 () {60 vars in 1144 bytes} [Thu Jul 14 09:08:16 2022] POST /identity/v3/endpoints => generated 331 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 09:08:26.288345 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=30 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:08:26.468004 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:09:18.901068 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230f3b8b-9815-47f4-b626-e7a2025d400f None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:09:18.901662 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230f3b8b-9815-47f4-b626-e7a2025d400f None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:09:18.901662 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230f3b8b-9815-47f4-b626-e7a2025d400f None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:09:18.906927 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 648/1307] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:09:18 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:09:18.908388 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073e06e2-1f0c-4905-baad-d69decc90bf6 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:09:18.908844 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073e06e2-1f0c-4905-baad-d69decc90bf6 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:09:18.908942 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073e06e2-1f0c-4905-baad-d69decc90bf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:09:18.980164 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-073e06e2-1f0c-4905-baad-d69decc90bf6 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:09:18.981380 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-073e06e2-1f0c-4905-baad-d69decc90bf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:09:19.161928 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 660/1308] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:09:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:09:19.219251 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9cf3e2-f31d-4d39-a052-9aa8b29dc957 None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:09:19.219664 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9cf3e2-f31d-4d39-a052-9aa8b29dc957 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:09:19.220199 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9cf3e2-f31d-4d39-a052-9aa8b29dc957 None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:09:19.222280 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 649/1309] 10.222.0.26 () {54 vars in 964 bytes} [Thu Jul 14 09:09:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:09:19.229421 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e848db-79c4-47ce-9a65-0a7c93bd3ca7 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:09:19.229718 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e848db-79c4-47ce-9a65-0a7c93bd3ca7 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:09:19.230033 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e848db-79c4-47ce-9a65-0a7c93bd3ca7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:09:19.328932 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-d2e848db-79c4-47ce-9a65-0a7c93bd3ca7 None None] MFA Rules not processed for user `952911c492444ffea4b211a24b564117`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:09:19.332000 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-d2e848db-79c4-47ce-9a65-0a7c93bd3ca7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:09:19.355619 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:09:19.413869 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 661/1310] 10.222.0.26 () {58 vars in 1061 bytes} [Thu Jul 14 09:09:19 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:09:19.422795 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9cf3e2-f31d-4d39-a052-9aa8b29dc957 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:09:19.424530 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-ab9cf3e2-f31d-4d39-a052-9aa8b29dc957 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:09:19.543466 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9cf3e2-f31d-4d39-a052-9aa8b29dc957 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:09:19.626009 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a90775-9461-4e93-971c-7c9db7df877c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '952911c492444ffea4b211a24b564117', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:09:19.626570 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a90775-9461-4e93-971c-7c9db7df877c service neutron] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:09:19.626832 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a90775-9461-4e93-971c-7c9db7df877c service neutron] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:09:19.627104 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a90775-9461-4e93-971c-7c9db7df877c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:09:19.628761 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-01a90775-9461-4e93-971c-7c9db7df877c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:09:19.684953 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:09:19.741376 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a90775-9461-4e93-971c-7c9db7df877c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:09:19.743094 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a90775-9461-4e93-971c-7c9db7df877c service neutron] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:09:19.763421 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 650/1311] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:09:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:09:22.041269 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b658381-d4b3-4e5e-9a12-c625a1b1da89 None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:09:22.041872 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b658381-d4b3-4e5e-9a12-c625a1b1da89 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:09:22.041872 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b658381-d4b3-4e5e-9a12-c625a1b1da89 None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:09:22.043537 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 662/1312] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:09:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:09:22.048432 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567c7948-db79-4131-8468-ba87cc563c8d None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:09:22.048855 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567c7948-db79-4131-8468-ba87cc563c8d None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:09:22.048962 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567c7948-db79-4131-8468-ba87cc563c8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:09:22.131204 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-567c7948-db79-4131-8468-ba87cc563c8d None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:09:22.132552 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-567c7948-db79-4131-8468-ba87cc563c8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:09:22.239280 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 651/1313] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:09:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:09:22.277997 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b658381-d4b3-4e5e-9a12-c625a1b1da89 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:09:22.279773 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-8b658381-d4b3-4e5e-9a12-c625a1b1da89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:09:22.398767 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b658381-d4b3-4e5e-9a12-c625a1b1da89 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:09:22.472535 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:09:22.475474 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f083a024-b477-4d83-87d2-12bbcecfe690 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '952911c492444ffea4b211a24b564117', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:09:22.476140 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f083a024-b477-4d83-87d2-12bbcecfe690 service neutron] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:09:22.476464 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f083a024-b477-4d83-87d2-12bbcecfe690 service neutron] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:09:22.476762 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f083a024-b477-4d83-87d2-12bbcecfe690 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:09:22.478783 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-f083a024-b477-4d83-87d2-12bbcecfe690 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:09:22.598820 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f083a024-b477-4d83-87d2-12bbcecfe690 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:09:22.600320 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f083a024-b477-4d83-87d2-12bbcecfe690 service neutron] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:09:22.621508 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 663/1314] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:09:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:09:24.940211 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c0a3eb-bfe2-4b45-9720-a727157e428c None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:09:24.941032 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c0a3eb-bfe2-4b45-9720-a727157e428c None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:09:24.941032 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c0a3eb-bfe2-4b45-9720-a727157e428c None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:09:24.944023 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 652/1315] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:09:24 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:09:24.949530 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69171f56-4b44-4cbc-a736-5fe33eee5db5 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:09:24.949985 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69171f56-4b44-4cbc-a736-5fe33eee5db5 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:09:24.950077 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69171f56-4b44-4cbc-a736-5fe33eee5db5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:09:25.018365 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-69171f56-4b44-4cbc-a736-5fe33eee5db5 None None] MFA Rules not processed for user `cb553a2f9c234bcbadbe3b9dbdd4a92e`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:09:25.019571 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-69171f56-4b44-4cbc-a736-5fe33eee5db5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:09:25.074139 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:09:25.123527 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 664/1316] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:09:24 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:09:25.186847 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e70f13a-468e-408f-89f2-c96d01518c8c None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:09:25.187152 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e70f13a-468e-408f-89f2-c96d01518c8c None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:09:25.187540 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e70f13a-468e-408f-89f2-c96d01518c8c None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:09:25.189730 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 653/1317] 10.222.0.26 () {54 vars in 964 bytes} [Thu Jul 14 09:09:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:09:25.195911 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be24e1c-524a-4b43-9f29-9cc44551d5fa None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:09:25.196140 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be24e1c-524a-4b43-9f29-9cc44551d5fa None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:09:25.196418 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be24e1c-524a-4b43-9f29-9cc44551d5fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:09:25.275494 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-5be24e1c-524a-4b43-9f29-9cc44551d5fa None None] MFA Rules not processed for user `952911c492444ffea4b211a24b564117`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:09:25.276723 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-5be24e1c-524a-4b43-9f29-9cc44551d5fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:09:25.356970 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 665/1318] 10.222.0.26 () {58 vars in 1061 bytes} [Thu Jul 14 09:09:25 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:09:25.366219 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e70f13a-468e-408f-89f2-c96d01518c8c None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:09:25.367826 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-0e70f13a-468e-408f-89f2-c96d01518c8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:09:25.378229 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:09:25.483578 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e70f13a-468e-408f-89f2-c96d01518c8c None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:09:25.564989 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ba5dd3-347f-432b-b1be-8ed567ac5d4f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '952911c492444ffea4b211a24b564117', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:09:25.565935 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ba5dd3-347f-432b-b1be-8ed567ac5d4f service neutron] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:09:25.566295 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ba5dd3-347f-432b-b1be-8ed567ac5d4f service neutron] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:09:25.566622 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ba5dd3-347f-432b-b1be-8ed567ac5d4f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:09:25.569264 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-f8ba5dd3-347f-432b-b1be-8ed567ac5d4f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:09:25.634456 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:09:25.692319 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ba5dd3-347f-432b-b1be-8ed567ac5d4f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:09:25.693750 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ba5dd3-347f-432b-b1be-8ed567ac5d4f service neutron] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:09:25.712833 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 654/1319] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:09:25 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 09:09:28.625304 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f21f51-1804-4d8f-be68-74062812c4b2 None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:09:28.625778 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f21f51-1804-4d8f-be68-74062812c4b2 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:09:28.625876 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f21f51-1804-4d8f-be68-74062812c4b2 None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:09:28.627235 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 666/1320] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:09:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:09:28.635112 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a0f7e5-e8b5-4b03-a516-1c2d30dc23b6 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:09:28.635560 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a0f7e5-e8b5-4b03-a516-1c2d30dc23b6 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:09:28.635699 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a0f7e5-e8b5-4b03-a516-1c2d30dc23b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:09:28.709150 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-79a0f7e5-e8b5-4b03-a516-1c2d30dc23b6 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:09:28.711115 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-79a0f7e5-e8b5-4b03-a516-1c2d30dc23b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:09:28.819859 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 655/1321] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:09:28 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:09:28.858723 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f21f51-1804-4d8f-be68-74062812c4b2 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:09:28.861119 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-f6f21f51-1804-4d8f-be68-74062812c4b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:09:28.922038 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:09:28.985245 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f21f51-1804-4d8f-be68-74062812c4b2 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:09:29.060226 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a051bf68-43b9-4fd6-95b2-5b42d505f4ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '952911c492444ffea4b211a24b564117', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:09:29.060866 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a051bf68-43b9-4fd6-95b2-5b42d505f4ab service neutron] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:09:29.061147 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a051bf68-43b9-4fd6-95b2-5b42d505f4ab service neutron] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:09:29.061422 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a051bf68-43b9-4fd6-95b2-5b42d505f4ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:09:29.063052 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-a051bf68-43b9-4fd6-95b2-5b42d505f4ab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:09:29.161850 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:09:29.175575 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a051bf68-43b9-4fd6-95b2-5b42d505f4ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:09:29.177026 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a051bf68-43b9-4fd6-95b2-5b42d505f4ab service neutron] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:09:29.195681 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 667/1322] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:09:28 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:09:31.946103 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213f5343-21d4-4ffa-a246-56bff25644e6 None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:09:31.946653 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213f5343-21d4-4ffa-a246-56bff25644e6 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:09:31.946653 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213f5343-21d4-4ffa-a246-56bff25644e6 None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:09:31.947982 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 656/1323] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:09:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:09:31.953520 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad4ac2f-1ac4-4018-ad09-2f00ce5664cd None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:09:31.953962 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad4ac2f-1ac4-4018-ad09-2f00ce5664cd None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:09:31.954073 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad4ac2f-1ac4-4018-ad09-2f00ce5664cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:09:32.023085 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-aad4ac2f-1ac4-4018-ad09-2f00ce5664cd None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:09:32.024460 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-aad4ac2f-1ac4-4018-ad09-2f00ce5664cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:09:32.133112 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 668/1324] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:09:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:09:32.169199 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213f5343-21d4-4ffa-a246-56bff25644e6 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:09:32.170775 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-213f5343-21d4-4ffa-a246-56bff25644e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:09:32.192766 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:09:32.293375 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213f5343-21d4-4ffa-a246-56bff25644e6 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:09:32.369089 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6baa5420-f194-4689-aaba-e482315fa04a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '952911c492444ffea4b211a24b564117', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:09:32.369739 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baa5420-f194-4689-aaba-e482315fa04a service neutron] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:09:32.370019 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baa5420-f194-4689-aaba-e482315fa04a service neutron] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:09:32.370289 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baa5420-f194-4689-aaba-e482315fa04a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:09:32.372247 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-6baa5420-f194-4689-aaba-e482315fa04a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:09:32.444057 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:09:32.481880 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6baa5420-f194-4689-aaba-e482315fa04a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:09:32.483365 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6baa5420-f194-4689-aaba-e482315fa04a service neutron] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:09:32.501589 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 657/1325] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:09:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:09:34.560527 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7351866a-62e7-44ec-857b-17180577cdb3 None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:09:34.560998 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7351866a-62e7-44ec-857b-17180577cdb3 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:09:34.561121 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7351866a-62e7-44ec-857b-17180577cdb3 None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:09:34.562462 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 669/1326] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:09:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:09:34.568006 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498b0232-668e-491d-86d5-24c39d867269 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:09:34.568444 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498b0232-668e-491d-86d5-24c39d867269 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:09:34.568607 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498b0232-668e-491d-86d5-24c39d867269 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:09:34.635976 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-498b0232-668e-491d-86d5-24c39d867269 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:09:34.637194 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-498b0232-668e-491d-86d5-24c39d867269 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:09:34.739687 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 658/1327] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:09:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:09:34.775885 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7351866a-62e7-44ec-857b-17180577cdb3 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:09:34.777401 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-7351866a-62e7-44ec-857b-17180577cdb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:09:34.840018 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:09:34.886566 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7351866a-62e7-44ec-857b-17180577cdb3 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:09:34.965933 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73ab792-7822-4d4b-940b-00ba53f62839 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '952911c492444ffea4b211a24b564117', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:09:34.966592 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73ab792-7822-4d4b-940b-00ba53f62839 service neutron] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:09:34.966888 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73ab792-7822-4d4b-940b-00ba53f62839 service neutron] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:09:34.967157 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73ab792-7822-4d4b-940b-00ba53f62839 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:09:34.968833 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-a73ab792-7822-4d4b-940b-00ba53f62839 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:09:35.080445 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:09:35.083002 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73ab792-7822-4d4b-940b-00ba53f62839 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:09:35.084879 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73ab792-7822-4d4b-940b-00ba53f62839 service neutron] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:09:35.110874 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 670/1328] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:09:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:09:37.152139 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14a1131-2a56-45fe-88a1-6f25b9136f34 None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:09:37.152697 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14a1131-2a56-45fe-88a1-6f25b9136f34 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:09:37.152697 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14a1131-2a56-45fe-88a1-6f25b9136f34 None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:09:37.154073 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 659/1329] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:09:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:09:37.159363 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6fd6b1-e6fc-41e0-8721-aa287fe3b118 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:09:37.159816 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6fd6b1-e6fc-41e0-8721-aa287fe3b118 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:09:37.159913 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6fd6b1-e6fc-41e0-8721-aa287fe3b118 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:09:37.231840 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-be6fd6b1-e6fc-41e0-8721-aa287fe3b118 None None] MFA Rules not processed for user `cb553a2f9c234bcbadbe3b9dbdd4a92e`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:09:37.233051 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-be6fd6b1-e6fc-41e0-8721-aa287fe3b118 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:09:37.336395 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 671/1330] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:09:37 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:09:37.372946 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14a1131-2a56-45fe-88a1-6f25b9136f34 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:09:37.374522 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-f14a1131-2a56-45fe-88a1-6f25b9136f34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:09:37.407563 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:09:37.484141 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14a1131-2a56-45fe-88a1-6f25b9136f34 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:09:37.562034 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62202307-27f8-4ab6-b66f-cb65f1c0aa9c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '952911c492444ffea4b211a24b564117', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:09:37.562745 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62202307-27f8-4ab6-b66f-cb65f1c0aa9c service neutron] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:09:37.563027 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62202307-27f8-4ab6-b66f-cb65f1c0aa9c service neutron] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:09:37.563343 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62202307-27f8-4ab6-b66f-cb65f1c0aa9c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:09:37.564980 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-62202307-27f8-4ab6-b66f-cb65f1c0aa9c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:09:37.656079 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:09:37.696599 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62202307-27f8-4ab6-b66f-cb65f1c0aa9c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:09:37.698047 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62202307-27f8-4ab6-b66f-cb65f1c0aa9c service neutron] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:09:37.717699 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 660/1331] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:09:37 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:09:39.982715 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b727fb9-f0a3-46a4-95f0-978d7f81d838 None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:09:39.983374 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b727fb9-f0a3-46a4-95f0-978d7f81d838 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:09:39.983374 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b727fb9-f0a3-46a4-95f0-978d7f81d838 None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:09:39.984717 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 672/1332] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:09:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:09:39.991033 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317d5553-6bd7-4859-8326-fb14ba8441a3 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:09:39.991774 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317d5553-6bd7-4859-8326-fb14ba8441a3 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:09:39.991957 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317d5553-6bd7-4859-8326-fb14ba8441a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:09:40.061325 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-317d5553-6bd7-4859-8326-fb14ba8441a3 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:09:40.062520 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-317d5553-6bd7-4859-8326-fb14ba8441a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:09:40.167452 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 661/1333] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:09:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:09:40.209792 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b727fb9-f0a3-46a4-95f0-978d7f81d838 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:09:40.211426 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-5b727fb9-f0a3-46a4-95f0-978d7f81d838 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:09:40.293462 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:09:40.333240 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b727fb9-f0a3-46a4-95f0-978d7f81d838 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:09:40.414640 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28f33b0-c3fa-482e-a9ac-e74598d37518 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '952911c492444ffea4b211a24b564117', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:09:40.415319 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28f33b0-c3fa-482e-a9ac-e74598d37518 service neutron] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:09:40.415605 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28f33b0-c3fa-482e-a9ac-e74598d37518 service neutron] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:09:40.415873 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28f33b0-c3fa-482e-a9ac-e74598d37518 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:09:40.417551 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-e28f33b0-c3fa-482e-a9ac-e74598d37518 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:09:40.536785 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28f33b0-c3fa-482e-a9ac-e74598d37518 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:09:40.538235 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28f33b0-c3fa-482e-a9ac-e74598d37518 service neutron] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:09:40.548858 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:09:40.558884 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 673/1334] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:09:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:09:42.632442 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb9038f-1547-433b-b929-83cab0651b18 None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:09:42.632909 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb9038f-1547-433b-b929-83cab0651b18 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:09:42.633069 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb9038f-1547-433b-b929-83cab0651b18 None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:09:42.634502 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 662/1335] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:09:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:09:42.642839 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d303b0b2-0bd5-43f6-8e69-178968a41001 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:09:42.643301 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d303b0b2-0bd5-43f6-8e69-178968a41001 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:09:42.643456 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d303b0b2-0bd5-43f6-8e69-178968a41001 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:09:42.717587 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-d303b0b2-0bd5-43f6-8e69-178968a41001 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:09:42.718797 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-d303b0b2-0bd5-43f6-8e69-178968a41001 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:09:42.819913 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 674/1336] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:09:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:09:42.862552 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb9038f-1547-433b-b929-83cab0651b18 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:09:42.865005 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-bdb9038f-1547-433b-b929-83cab0651b18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:09:42.913182 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:09:42.979292 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb9038f-1547-433b-b929-83cab0651b18 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:09:43.055543 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea9321e-382b-4baf-b0ea-8033324f38ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '952911c492444ffea4b211a24b564117', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:09:43.056190 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea9321e-382b-4baf-b0ea-8033324f38ba service neutron] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:09:43.056475 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea9321e-382b-4baf-b0ea-8033324f38ba service neutron] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:09:43.056749 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea9321e-382b-4baf-b0ea-8033324f38ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:09:43.058378 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-9ea9321e-382b-4baf-b0ea-8033324f38ba service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:09:43.147258 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:09:43.174829 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea9321e-382b-4baf-b0ea-8033324f38ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:09:43.176658 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea9321e-382b-4baf-b0ea-8033324f38ba service neutron] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:09:43.197038 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 663/1337] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:09:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:09:45.548964 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a186895c-c956-45c0-84c8-0eecb3adaf6e None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:09:45.549518 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a186895c-c956-45c0-84c8-0eecb3adaf6e None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:09:45.549518 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a186895c-c956-45c0-84c8-0eecb3adaf6e None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:09:45.550862 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 675/1338] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:09:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:09:45.557497 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220d72ec-f368-42a0-8404-f9d200c99eb4 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:09:45.557964 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220d72ec-f368-42a0-8404-f9d200c99eb4 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:09:45.558062 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220d72ec-f368-42a0-8404-f9d200c99eb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:09:45.630492 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-220d72ec-f368-42a0-8404-f9d200c99eb4 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:09:45.631771 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-220d72ec-f368-42a0-8404-f9d200c99eb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:09:45.732346 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 664/1339] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:09:45 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:09:45.781352 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a186895c-c956-45c0-84c8-0eecb3adaf6e None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:09:45.782907 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-a186895c-c956-45c0-84c8-0eecb3adaf6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:09:45.872559 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:09:45.902817 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a186895c-c956-45c0-84c8-0eecb3adaf6e None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:09:45.977932 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e31898-1f42-46b0-9793-464668d82c2b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '952911c492444ffea4b211a24b564117', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:09:45.978585 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e31898-1f42-46b0-9793-464668d82c2b service neutron] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:09:45.978840 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e31898-1f42-46b0-9793-464668d82c2b service neutron] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:09:45.979139 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e31898-1f42-46b0-9793-464668d82c2b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:09:45.980809 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-64e31898-1f42-46b0-9793-464668d82c2b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:09:46.102767 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e31898-1f42-46b0-9793-464668d82c2b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:09:46.104253 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e31898-1f42-46b0-9793-464668d82c2b service neutron] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:09:46.123969 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 676/1340] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:09:45 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:09:50.108943 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500d3c50-f5cb-496d-b4b8-e8b17ce7faa4 None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:09:50.109490 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500d3c50-f5cb-496d-b4b8-e8b17ce7faa4 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:09:50.109490 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500d3c50-f5cb-496d-b4b8-e8b17ce7faa4 None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:09:50.110804 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 665/1341] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:09:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:09:50.116268 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c82ed5-2fcc-4481-b5cf-e2785c91776c None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:09:50.116725 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c82ed5-2fcc-4481-b5cf-e2785c91776c None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:09:50.116938 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c82ed5-2fcc-4481-b5cf-e2785c91776c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:09:50.122006 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:09:50.185932 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-19c82ed5-2fcc-4481-b5cf-e2785c91776c None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:09:50.187137 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-19c82ed5-2fcc-4481-b5cf-e2785c91776c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:09:50.286327 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 677/1342] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:09:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:09:50.323914 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500d3c50-f5cb-496d-b4b8-e8b17ce7faa4 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:09:50.325452 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-500d3c50-f5cb-496d-b4b8-e8b17ce7faa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:09:50.383917 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:09:50.440467 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500d3c50-f5cb-496d-b4b8-e8b17ce7faa4 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:09:50.517507 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b11c248-3c8f-4833-b262-08b96717e1d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '952911c492444ffea4b211a24b564117', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:09:50.518262 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b11c248-3c8f-4833-b262-08b96717e1d5 service neutron] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:09:50.518538 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b11c248-3c8f-4833-b262-08b96717e1d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:09:50.518805 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b11c248-3c8f-4833-b262-08b96717e1d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:09:50.520457 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-1b11c248-3c8f-4833-b262-08b96717e1d5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:09:50.620960 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:09:50.639889 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b11c248-3c8f-4833-b262-08b96717e1d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:09:50.641355 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b11c248-3c8f-4833-b262-08b96717e1d5 service neutron] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:09:50.661337 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 666/1343] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:09:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:09:53.454771 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecccd16-5da7-4546-a5e8-16fa3a81b41d None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:09:53.455394 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecccd16-5da7-4546-a5e8-16fa3a81b41d None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:09:53.455394 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecccd16-5da7-4546-a5e8-16fa3a81b41d None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:09:53.456755 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 678/1344] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:09:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:09:53.462213 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37597c8a-08f8-4375-8772-335f254d0f43 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:09:53.462640 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37597c8a-08f8-4375-8772-335f254d0f43 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:09:53.462842 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37597c8a-08f8-4375-8772-335f254d0f43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:09:53.535236 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-37597c8a-08f8-4375-8772-335f254d0f43 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:09:53.536479 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-37597c8a-08f8-4375-8772-335f254d0f43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:09:53.636106 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 667/1345] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:09:53 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:09:53.673432 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecccd16-5da7-4546-a5e8-16fa3a81b41d None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:09:53.674978 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-0ecccd16-5da7-4546-a5e8-16fa3a81b41d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:09:53.765449 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:09:53.783178 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecccd16-5da7-4546-a5e8-16fa3a81b41d None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:09:53.860403 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2332b5d2-5dfe-40d2-92a9-c9a151bfce76 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '952911c492444ffea4b211a24b564117', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:09:53.860975 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2332b5d2-5dfe-40d2-92a9-c9a151bfce76 service neutron] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:09:53.861253 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2332b5d2-5dfe-40d2-92a9-c9a151bfce76 service neutron] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:09:53.861522 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2332b5d2-5dfe-40d2-92a9-c9a151bfce76 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:09:53.863220 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-2332b5d2-5dfe-40d2-92a9-c9a151bfce76 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:09:53.982822 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2332b5d2-5dfe-40d2-92a9-c9a151bfce76 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:09:53.984301 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2332b5d2-5dfe-40d2-92a9-c9a151bfce76 service neutron] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:09:54.004066 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 679/1346] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:09:53 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:09:57.935344 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ae1de2-b7cd-48fd-897c-4f256b6fd7c4 None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:09:57.935898 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ae1de2-b7cd-48fd-897c-4f256b6fd7c4 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:09:57.936123 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ae1de2-b7cd-48fd-897c-4f256b6fd7c4 None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:09:57.937944 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 668/1347] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:09:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:09:57.943922 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f42c53-7af2-4ec2-852e-0c3667b7182d None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:09:57.944439 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f42c53-7af2-4ec2-852e-0c3667b7182d None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:09:57.944439 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f42c53-7af2-4ec2-852e-0c3667b7182d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:09:57.960303 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:09:58.015954 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-37f42c53-7af2-4ec2-852e-0c3667b7182d None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:09:58.017140 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-37f42c53-7af2-4ec2-852e-0c3667b7182d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:09:58.117308 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 680/1348] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:09:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 09:09:58.155298 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ae1de2-b7cd-48fd-897c-4f256b6fd7c4 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:09:58.156874 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-26ae1de2-b7cd-48fd-897c-4f256b6fd7c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:09:58.217363 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:09:58.267436 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ae1de2-b7cd-48fd-897c-4f256b6fd7c4 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:09:58.344327 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d687e1f9-fdde-4309-a304-ce612ac077db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '952911c492444ffea4b211a24b564117', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:09:58.345191 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d687e1f9-fdde-4309-a304-ce612ac077db service neutron] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:09:58.345501 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d687e1f9-fdde-4309-a304-ce612ac077db service neutron] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:09:58.345780 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d687e1f9-fdde-4309-a304-ce612ac077db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:09:58.347545 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-d687e1f9-fdde-4309-a304-ce612ac077db service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:09:58.456947 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:09:58.461498 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d687e1f9-fdde-4309-a304-ce612ac077db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:09:58.463332 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d687e1f9-fdde-4309-a304-ce612ac077db service neutron] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:09:58.484281 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 669/1349] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:09:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:10:03.446946 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317d8d67-aa5e-46f8-9c63-3962c5db1b36 None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:10:03.463313 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317d8d67-aa5e-46f8-9c63-3962c5db1b36 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:10:03.463313 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317d8d67-aa5e-46f8-9c63-3962c5db1b36 None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:10:03.463313 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 681/1350] 10.222.0.26 () {54 vars in 921 bytes} [Thu Jul 14 09:10:03 2022] GET /identity => generated 270 bytes in 18 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:10:03.479897 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22578d2f-1812-4ead-aca1-3828cd128f76 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:10:03.481645 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22578d2f-1812-4ead-aca1-3828cd128f76 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:10:03.482377 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22578d2f-1812-4ead-aca1-3828cd128f76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:10:03.582690 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-22578d2f-1812-4ead-aca1-3828cd128f76 None None] MFA Rules not processed for user `355684f52e074ad9bfa7fd90bcae6caf`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:10:03.584040 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-22578d2f-1812-4ead-aca1-3828cd128f76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:10:03.702122 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 670/1351] 10.222.0.26 () {58 vars in 1018 bytes} [Thu Jul 14 09:10:03 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:10:07.680132 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fffba9-4572-49aa-84f2-44ed82fc928c None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:10:07.681084 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fffba9-4572-49aa-84f2-44ed82fc928c None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:10:07.681533 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fffba9-4572-49aa-84f2-44ed82fc928c None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:10:07.683102 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 682/1352] 10.222.0.26 () {54 vars in 921 bytes} [Thu Jul 14 09:10:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:10:07.688189 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cd9fec-0bd5-469b-8212-ec25c36e175b None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:10:07.688592 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cd9fec-0bd5-469b-8212-ec25c36e175b None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:10:07.688914 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cd9fec-0bd5-469b-8212-ec25c36e175b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:10:07.749264 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:10:07.774180 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-c0cd9fec-0bd5-469b-8212-ec25c36e175b None None] MFA Rules not processed for user `355684f52e074ad9bfa7fd90bcae6caf`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:10:07.775386 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-c0cd9fec-0bd5-469b-8212-ec25c36e175b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:10:07.887310 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 671/1353] 10.222.0.26 () {58 vars in 1018 bytes} [Thu Jul 14 09:10:07 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:10:08.115430 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=59 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:10:09.369991 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43e5cdd-a728-464a-80da-c90ae1ec8f2d None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:10:09.370649 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43e5cdd-a728-464a-80da-c90ae1ec8f2d None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:10:09.370649 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43e5cdd-a728-464a-80da-c90ae1ec8f2d None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:10:09.372149 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 683/1354] 10.222.0.26 () {54 vars in 921 bytes} [Thu Jul 14 09:10:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:10:09.377312 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b4d5e7-09c8-4b88-b1e3-f8cc148174a4 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:10:09.377772 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b4d5e7-09c8-4b88-b1e3-f8cc148174a4 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:10:09.377866 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b4d5e7-09c8-4b88-b1e3-f8cc148174a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:10:09.458965 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-d2b4d5e7-09c8-4b88-b1e3-f8cc148174a4 None None] MFA Rules not processed for user `355684f52e074ad9bfa7fd90bcae6caf`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:10:09.460222 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-d2b4d5e7-09c8-4b88-b1e3-f8cc148174a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:10:09.509145 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:10:09.589565 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 672/1355] 10.222.0.26 () {58 vars in 1018 bytes} [Thu Jul 14 09:10:09 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:10:12.811961 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c88da2-8122-4d83-a900-c7c456eac33e None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:10:12.812836 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c88da2-8122-4d83-a900-c7c456eac33e None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:10:12.813274 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c88da2-8122-4d83-a900-c7c456eac33e None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:10:12.819734 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 684/1356] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:10:12 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:10:12.821256 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f54c1b3-eff4-4f9c-9225-59a779509f2b None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:10:12.821705 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f54c1b3-eff4-4f9c-9225-59a779509f2b None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:10:12.821843 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f54c1b3-eff4-4f9c-9225-59a779509f2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:10:12.906645 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-0f54c1b3-eff4-4f9c-9225-59a779509f2b None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:10:12.908173 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-0f54c1b3-eff4-4f9c-9225-59a779509f2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:10:13.021105 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 673/1357] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:10:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:10:13.080693 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56c4cdf-8627-4fbf-a428-2559a54932ec None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:10:13.081399 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56c4cdf-8627-4fbf-a428-2559a54932ec None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:10:13.081840 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56c4cdf-8627-4fbf-a428-2559a54932ec None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:10:13.083448 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 685/1358] 10.222.0.26 () {54 vars in 961 bytes} [Thu Jul 14 09:10:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:10:13.092324 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e8af23-b639-4089-ba86-bffce5d44eea None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:10:13.092838 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e8af23-b639-4089-ba86-bffce5d44eea None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:10:13.093817 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e8af23-b639-4089-ba86-bffce5d44eea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:10:13.114595 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:10:13.180230 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-d8e8af23-b639-4089-ba86-bffce5d44eea None None] MFA Rules not processed for user `538fff3c5a2b487eb44ae7c1a1198102`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:10:13.181707 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-d8e8af23-b639-4089-ba86-bffce5d44eea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:10:13.323550 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 674/1359] 10.222.0.26 () {58 vars in 1058 bytes} [Thu Jul 14 09:10:13 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:10:13.331870 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56c4cdf-8627-4fbf-a428-2559a54932ec None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:10:13.333368 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-e56c4cdf-8627-4fbf-a428-2559a54932ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:10:13.507567 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56c4cdf-8627-4fbf-a428-2559a54932ec None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:10:13.593693 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:10:13.635770 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83647683-e199-412f-bda3-067f951cb332 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '538fff3c5a2b487eb44ae7c1a1198102', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:10:13.636424 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83647683-e199-412f-bda3-067f951cb332 service nova] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:10:13.636694 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83647683-e199-412f-bda3-067f951cb332 service nova] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:10:13.636985 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83647683-e199-412f-bda3-067f951cb332 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:10:13.638637 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-83647683-e199-412f-bda3-067f951cb332 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:10:13.734150 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83647683-e199-412f-bda3-067f951cb332 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:10:13.735923 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83647683-e199-412f-bda3-067f951cb332 service nova] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:10:13.755172 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 686/1360] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:10:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:10:15.969941 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89098a6-f2d8-4140-a632-b8cc43dad3b0 None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:10:15.970515 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89098a6-f2d8-4140-a632-b8cc43dad3b0 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:10:15.970741 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89098a6-f2d8-4140-a632-b8cc43dad3b0 None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:10:15.977722 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fff78a0-7d9b-4b31-af5f-81efce1d2530 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:10:15.978234 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fff78a0-7d9b-4b31-af5f-81efce1d2530 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:10:15.978475 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fff78a0-7d9b-4b31-af5f-81efce1d2530 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:10:15.981758 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 675/1361] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:10:15 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:10:16.054467 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-9fff78a0-7d9b-4b31-af5f-81efce1d2530 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:10:16.055804 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-9fff78a0-7d9b-4b31-af5f-81efce1d2530 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:10:16.103501 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:10:16.171228 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 687/1362] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:10:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 09:10:16.215177 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89098a6-f2d8-4140-a632-b8cc43dad3b0 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:10:16.216628 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-f89098a6-f2d8-4140-a632-b8cc43dad3b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:10:16.295063 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:10:16.417260 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89098a6-f2d8-4140-a632-b8cc43dad3b0 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:10:16.552854 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0035f6-18f5-476b-929a-b44ae44df569 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '538fff3c5a2b487eb44ae7c1a1198102', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:10:16.553587 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0035f6-18f5-476b-929a-b44ae44df569 service nova] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:10:16.554021 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0035f6-18f5-476b-929a-b44ae44df569 service nova] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:10:16.554301 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0035f6-18f5-476b-929a-b44ae44df569 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:10:16.556076 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-5a0035f6-18f5-476b-929a-b44ae44df569 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:10:16.580333 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:10:16.650562 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0035f6-18f5-476b-929a-b44ae44df569 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:10:16.652144 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0035f6-18f5-476b-929a-b44ae44df569 service nova] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:10:16.671650 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 676/1363] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:10:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 457 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:10:19.029549 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeead9a3-94cb-492c-9ce8-c2df3d337c0d None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:10:19.030682 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeead9a3-94cb-492c-9ce8-c2df3d337c0d None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:10:19.030991 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeead9a3-94cb-492c-9ce8-c2df3d337c0d None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:10:19.032598 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 688/1364] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:10:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:10:19.038229 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edf8594-f490-41f2-ac1c-7af88ed046bc None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:10:19.038696 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edf8594-f490-41f2-ac1c-7af88ed046bc None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:10:19.038806 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edf8594-f490-41f2-ac1c-7af88ed046bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:10:19.114207 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-2edf8594-f490-41f2-ac1c-7af88ed046bc None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:10:19.115599 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-2edf8594-f490-41f2-ac1c-7af88ed046bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:10:19.223490 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:10:19.235049 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 677/1365] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:10:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:10:19.285391 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d468fb46-def5-4570-97cc-aca1b69ff101 None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:10:19.286245 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d468fb46-def5-4570-97cc-aca1b69ff101 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:10:19.286840 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d468fb46-def5-4570-97cc-aca1b69ff101 None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:10:19.289860 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 689/1366] 10.222.0.26 () {54 vars in 961 bytes} [Thu Jul 14 09:10:19 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:10:19.295965 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fccff58-eb0f-41c7-9905-ec82a14e7313 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:10:19.296255 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fccff58-eb0f-41c7-9905-ec82a14e7313 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:10:19.296539 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fccff58-eb0f-41c7-9905-ec82a14e7313 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:10:19.385112 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-5fccff58-eb0f-41c7-9905-ec82a14e7313 None None] MFA Rules not processed for user `538fff3c5a2b487eb44ae7c1a1198102`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:10:19.386390 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-5fccff58-eb0f-41c7-9905-ec82a14e7313 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:10:19.522439 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 678/1367] 10.222.0.26 () {58 vars in 1058 bytes} [Thu Jul 14 09:10:19 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 09:10:19.534997 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d468fb46-def5-4570-97cc-aca1b69ff101 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:10:19.534997 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-d468fb46-def5-4570-97cc-aca1b69ff101 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:10:19.700106 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d468fb46-def5-4570-97cc-aca1b69ff101 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:10:19.720706 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:10:19.826636 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef38e33-8f7f-4106-9340-4b1b715a41aa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '538fff3c5a2b487eb44ae7c1a1198102', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:10:19.827285 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef38e33-8f7f-4106-9340-4b1b715a41aa service nova] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:10:19.827546 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef38e33-8f7f-4106-9340-4b1b715a41aa service nova] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:10:19.827830 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef38e33-8f7f-4106-9340-4b1b715a41aa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:10:19.829484 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-4ef38e33-8f7f-4106-9340-4b1b715a41aa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:10:19.928620 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef38e33-8f7f-4106-9340-4b1b715a41aa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:10:19.930211 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef38e33-8f7f-4106-9340-4b1b715a41aa service nova] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:10:19.949245 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 690/1368] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:10:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:10:20.786147 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8316e893-2dde-4ce1-a4f1-357c3ac5a97d None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:10:20.786732 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8316e893-2dde-4ce1-a4f1-357c3ac5a97d None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:10:20.786732 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8316e893-2dde-4ce1-a4f1-357c3ac5a97d None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:10:20.788292 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 679/1369] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:10:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:10:20.793569 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e385684-2c6e-4f4c-830f-96598a5d9e26 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:10:20.794000 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e385684-2c6e-4f4c-830f-96598a5d9e26 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:10:20.794180 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e385684-2c6e-4f4c-830f-96598a5d9e26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:10:20.826198 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:10:20.862408 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-6e385684-2c6e-4f4c-830f-96598a5d9e26 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:10:20.863741 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-6e385684-2c6e-4f4c-830f-96598a5d9e26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:10:20.963985 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 691/1370] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:10:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:10:21.269865 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d561d6-6ab1-4886-80e2-4674f8abb57d None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:10:21.270130 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d561d6-6ab1-4886-80e2-4674f8abb57d None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:10:21.270408 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d561d6-6ab1-4886-80e2-4674f8abb57d None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:10:21.272115 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 680/1371] 10.222.0.26 () {54 vars in 963 bytes} [Thu Jul 14 09:10:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:10:21.277722 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b447491-2e1d-42a5-be15-a75aeb8fd6ed None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:10:21.278015 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b447491-2e1d-42a5-be15-a75aeb8fd6ed None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:10:21.278366 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b447491-2e1d-42a5-be15-a75aeb8fd6ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:10:21.360465 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-0b447491-2e1d-42a5-be15-a75aeb8fd6ed None None] MFA Rules not processed for user `e76f1952a5b146808844b9f2b7395035`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:10:21.361677 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-0b447491-2e1d-42a5-be15-a75aeb8fd6ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:10:21.411519 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:10:21.447295 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 692/1372] 10.222.0.26 () {58 vars in 1060 bytes} [Thu Jul 14 09:10:21 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:10:21.454593 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d561d6-6ab1-4886-80e2-4674f8abb57d None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:10:21.456131 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-60d561d6-6ab1-4886-80e2-4674f8abb57d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:10:21.509371 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:10:21.580232 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d561d6-6ab1-4886-80e2-4674f8abb57d None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:10:21.664628 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97018389-e4d7-4e16-abfe-0ea8e1667c79 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e76f1952a5b146808844b9f2b7395035', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:10:21.665574 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97018389-e4d7-4e16-abfe-0ea8e1667c79 service cinder] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:10:21.665952 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97018389-e4d7-4e16-abfe-0ea8e1667c79 service cinder] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:10:21.666358 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97018389-e4d7-4e16-abfe-0ea8e1667c79 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:10:21.668117 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-97018389-e4d7-4e16-abfe-0ea8e1667c79 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:10:21.766914 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:10:21.782243 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97018389-e4d7-4e16-abfe-0ea8e1667c79 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:10:21.783882 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97018389-e4d7-4e16-abfe-0ea8e1667c79 service cinder] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:10:21.803329 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 681/1373] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:10:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:10:22.018295 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f49f42-dc88-4f05-b7f3-8eecc6d81bf4 None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:10:22.018833 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f49f42-dc88-4f05-b7f3-8eecc6d81bf4 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:10:22.018833 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f49f42-dc88-4f05-b7f3-8eecc6d81bf4 None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:10:22.020625 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 693/1374] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:10:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:10:22.025888 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c641724-893c-46b9-9bed-4a1ce38224b9 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:10:22.026289 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c641724-893c-46b9-9bed-4a1ce38224b9 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:10:22.026673 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c641724-893c-46b9-9bed-4a1ce38224b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:10:22.104279 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-9c641724-893c-46b9-9bed-4a1ce38224b9 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:10:22.105607 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-9c641724-893c-46b9-9bed-4a1ce38224b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:10:22.223152 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 682/1375] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:10:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:10:22.267019 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f49f42-dc88-4f05-b7f3-8eecc6d81bf4 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:10:22.268562 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-c5f49f42-dc88-4f05-b7f3-8eecc6d81bf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:10:22.419551 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f49f42-dc88-4f05-b7f3-8eecc6d81bf4 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:10:22.470643 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:10:22.542281 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e6a1fc-f264-4f72-84d4-458671b8b306 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '538fff3c5a2b487eb44ae7c1a1198102', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:10:22.542281 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e6a1fc-f264-4f72-84d4-458671b8b306 service nova] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:10:22.543035 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e6a1fc-f264-4f72-84d4-458671b8b306 service nova] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:10:22.543035 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e6a1fc-f264-4f72-84d4-458671b8b306 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:10:22.544720 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-83e6a1fc-f264-4f72-84d4-458671b8b306 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:10:22.638340 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e6a1fc-f264-4f72-84d4-458671b8b306 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:10:22.639981 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e6a1fc-f264-4f72-84d4-458671b8b306 service nova] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:10:22.658221 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 694/1376] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:10:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:10:24.675241 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da702b59-dcc7-46b5-8ae4-744522723c02 None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:10:24.675717 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da702b59-dcc7-46b5-8ae4-744522723c02 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:10:24.675808 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da702b59-dcc7-46b5-8ae4-744522723c02 None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:10:24.677199 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 683/1377] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:10:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:10:24.682610 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1c540a-0f90-46fa-aff0-2e75b5153af1 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:10:24.683063 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1c540a-0f90-46fa-aff0-2e75b5153af1 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:10:24.683169 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1c540a-0f90-46fa-aff0-2e75b5153af1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:10:24.750704 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-be1c540a-0f90-46fa-aff0-2e75b5153af1 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:10:24.751960 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-be1c540a-0f90-46fa-aff0-2e75b5153af1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:10:24.763988 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:10:24.863694 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 695/1378] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:10:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:10:24.907715 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da702b59-dcc7-46b5-8ae4-744522723c02 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:10:24.909217 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-da702b59-dcc7-46b5-8ae4-744522723c02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:10:24.973948 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:10:25.060939 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da702b59-dcc7-46b5-8ae4-744522723c02 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:10:25.195867 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d9d62c-823f-4280-a550-dbbb9c24164c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '538fff3c5a2b487eb44ae7c1a1198102', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:10:25.196629 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d9d62c-823f-4280-a550-dbbb9c24164c service nova] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:10:25.196907 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d9d62c-823f-4280-a550-dbbb9c24164c service nova] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:10:25.197181 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d9d62c-823f-4280-a550-dbbb9c24164c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:10:25.198807 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-e6d9d62c-823f-4280-a550-dbbb9c24164c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:10:25.218716 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:10:25.313482 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d9d62c-823f-4280-a550-dbbb9c24164c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:10:25.315449 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d9d62c-823f-4280-a550-dbbb9c24164c service nova] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:10:25.339477 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 684/1379] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:10:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:10:27.391118 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92098736-8629-4a4d-8471-f808960b1519 None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:10:27.391676 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92098736-8629-4a4d-8471-f808960b1519 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:10:27.391797 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92098736-8629-4a4d-8471-f808960b1519 None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:10:27.393130 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 696/1380] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:10:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:10:27.398717 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5f78f9-b5e1-4ee1-b3b8-e941e15cba80 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:10:27.399237 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5f78f9-b5e1-4ee1-b3b8-e941e15cba80 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:10:27.399237 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5f78f9-b5e1-4ee1-b3b8-e941e15cba80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:10:27.465895 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-6e5f78f9-b5e1-4ee1-b3b8-e941e15cba80 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:10:27.467083 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-6e5f78f9-b5e1-4ee1-b3b8-e941e15cba80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:10:27.572393 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:10:27.583326 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 685/1381] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:10:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:10:27.627630 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92098736-8629-4a4d-8471-f808960b1519 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:10:27.629342 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-92098736-8629-4a4d-8471-f808960b1519 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:10:27.778375 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:10:27.789752 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92098736-8629-4a4d-8471-f808960b1519 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:10:27.909002 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75aa3a0b-5f9b-483f-85fa-ccb7276d0853 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '538fff3c5a2b487eb44ae7c1a1198102', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:10:27.909560 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75aa3a0b-5f9b-483f-85fa-ccb7276d0853 service nova] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:10:27.909845 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75aa3a0b-5f9b-483f-85fa-ccb7276d0853 service nova] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:10:27.910112 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75aa3a0b-5f9b-483f-85fa-ccb7276d0853 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:10:27.911774 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-75aa3a0b-5f9b-483f-85fa-ccb7276d0853 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:10:28.005274 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75aa3a0b-5f9b-483f-85fa-ccb7276d0853 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:10:28.006989 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75aa3a0b-5f9b-483f-85fa-ccb7276d0853 service nova] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:10:28.017946 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:10:28.027698 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 697/1382] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:10:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:10:30.301002 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de48e93a-0b8a-4140-901b-7f79c7aac1f7 None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:10:30.301603 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de48e93a-0b8a-4140-901b-7f79c7aac1f7 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:10:30.301603 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de48e93a-0b8a-4140-901b-7f79c7aac1f7 None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:10:30.303165 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 686/1383] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:10:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 09:10:30.309178 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20b3cf1-7cd2-4a0f-949f-f873c6f9b26b None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:10:30.309739 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20b3cf1-7cd2-4a0f-949f-f873c6f9b26b None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:10:30.309739 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20b3cf1-7cd2-4a0f-949f-f873c6f9b26b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:10:30.384378 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-e20b3cf1-7cd2-4a0f-949f-f873c6f9b26b None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:10:30.385610 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-e20b3cf1-7cd2-4a0f-949f-f873c6f9b26b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:10:30.500203 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 698/1384] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:10:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:10:30.550293 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de48e93a-0b8a-4140-901b-7f79c7aac1f7 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:10:30.551944 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-de48e93a-0b8a-4140-901b-7f79c7aac1f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:10:30.713700 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de48e93a-0b8a-4140-901b-7f79c7aac1f7 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:10:30.785537 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:10:30.833589 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d5c597-e52f-4d5c-9d89-add036798e63 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '538fff3c5a2b487eb44ae7c1a1198102', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:10:30.834171 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d5c597-e52f-4d5c-9d89-add036798e63 service nova] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:10:30.834420 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d5c597-e52f-4d5c-9d89-add036798e63 service nova] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:10:30.834692 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d5c597-e52f-4d5c-9d89-add036798e63 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:10:30.836508 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-04d5c597-e52f-4d5c-9d89-add036798e63 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:10:30.945043 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d5c597-e52f-4d5c-9d89-add036798e63 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:10:30.946646 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d5c597-e52f-4d5c-9d89-add036798e63 service nova] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:10:30.966169 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 687/1385] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:10:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:10:32.965279 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c0705b-b61c-4b32-b5c1-1712bd7b2082 None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:10:32.965942 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c0705b-b61c-4b32-b5c1-1712bd7b2082 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:10:32.965942 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c0705b-b61c-4b32-b5c1-1712bd7b2082 None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:10:32.967232 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 699/1386] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:10:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 09:10:32.972655 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9d7199-154e-467f-9910-626ff7c7fd8c None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:10:32.973067 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9d7199-154e-467f-9910-626ff7c7fd8c None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:10:32.973288 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9d7199-154e-467f-9910-626ff7c7fd8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:10:33.041953 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-4d9d7199-154e-467f-9910-626ff7c7fd8c None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:10:33.043181 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-4d9d7199-154e-467f-9910-626ff7c7fd8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:10:33.076391 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:10:33.145430 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 688/1387] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:10:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:10:33.199233 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c0705b-b61c-4b32-b5c1-1712bd7b2082 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:10:33.200805 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-61c0705b-b61c-4b32-b5c1-1712bd7b2082 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:10:33.284727 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:10:33.375496 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c0705b-b61c-4b32-b5c1-1712bd7b2082 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:10:33.500942 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4305d6f-0c05-4970-bb3c-23c2fcf323f6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '538fff3c5a2b487eb44ae7c1a1198102', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:10:33.501593 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4305d6f-0c05-4970-bb3c-23c2fcf323f6 service nova] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:10:33.501875 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4305d6f-0c05-4970-bb3c-23c2fcf323f6 service nova] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:10:33.502146 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4305d6f-0c05-4970-bb3c-23c2fcf323f6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:10:33.503814 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-c4305d6f-0c05-4970-bb3c-23c2fcf323f6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:10:33.542845 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:10:33.596847 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4305d6f-0c05-4970-bb3c-23c2fcf323f6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:10:33.598513 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4305d6f-0c05-4970-bb3c-23c2fcf323f6 service nova] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:10:33.617727 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 700/1388] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:10:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:10:35.683654 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81349bc8-285b-4955-99a6-7b491814fed9 None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:10:35.684246 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81349bc8-285b-4955-99a6-7b491814fed9 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:10:35.684246 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81349bc8-285b-4955-99a6-7b491814fed9 None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:10:35.685581 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 689/1389] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:10:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:10:35.690983 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3b22f4-c54b-4da4-97e7-dce62940d954 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:10:35.691443 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3b22f4-c54b-4da4-97e7-dce62940d954 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:10:35.691538 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3b22f4-c54b-4da4-97e7-dce62940d954 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:10:35.759736 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-fd3b22f4-c54b-4da4-97e7-dce62940d954 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:10:35.761269 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-fd3b22f4-c54b-4da4-97e7-dce62940d954 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:10:35.860721 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 701/1390] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:10:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:10:35.904802 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81349bc8-285b-4955-99a6-7b491814fed9 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:10:35.906349 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-81349bc8-285b-4955-99a6-7b491814fed9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:10:36.059135 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81349bc8-285b-4955-99a6-7b491814fed9 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:10:36.076011 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:10:36.175645 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3c280b-3ff6-4014-927b-a3b5d0d1325a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '538fff3c5a2b487eb44ae7c1a1198102', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:10:36.176225 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3c280b-3ff6-4014-927b-a3b5d0d1325a service nova] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:10:36.176512 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3c280b-3ff6-4014-927b-a3b5d0d1325a service nova] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:10:36.176794 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3c280b-3ff6-4014-927b-a3b5d0d1325a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:10:36.178473 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-bc3c280b-3ff6-4014-927b-a3b5d0d1325a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:10:36.276741 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3c280b-3ff6-4014-927b-a3b5d0d1325a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:10:36.278679 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3c280b-3ff6-4014-927b-a3b5d0d1325a service nova] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:10:36.297549 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 690/1391] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:10:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:10:37.488993 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d7648c-b380-4f2a-a74b-19f2be88327c None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:10:37.489674 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d7648c-b380-4f2a-a74b-19f2be88327c None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:10:37.489674 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d7648c-b380-4f2a-a74b-19f2be88327c None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:10:37.490953 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 702/1392] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:10:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:10:37.496355 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47ddc29-60fe-4af9-bf5b-371cb5ae3b6f None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:10:37.496840 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47ddc29-60fe-4af9-bf5b-371cb5ae3b6f None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:10:37.496943 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47ddc29-60fe-4af9-bf5b-371cb5ae3b6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:10:37.522338 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:10:37.567801 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-c47ddc29-60fe-4af9-bf5b-371cb5ae3b6f None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:10:37.569021 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-c47ddc29-60fe-4af9-bf5b-371cb5ae3b6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:10:37.668471 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 691/1393] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:10:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 09:10:37.724286 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c78ebe-3dd1-498b-ab95-6b1bc3e56a41 None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:10:37.724680 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c78ebe-3dd1-498b-ab95-6b1bc3e56a41 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:10:37.725023 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c78ebe-3dd1-498b-ab95-6b1bc3e56a41 None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:10:37.726463 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 703/1394] 10.222.0.26 () {54 vars in 962 bytes} [Thu Jul 14 09:10:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:10:37.731976 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cef75b-7e39-45b0-b4b0-4e483bbd43a8 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:10:37.732269 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cef75b-7e39-45b0-b4b0-4e483bbd43a8 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:10:37.732543 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cef75b-7e39-45b0-b4b0-4e483bbd43a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:10:37.815013 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-58cef75b-7e39-45b0-b4b0-4e483bbd43a8 None None] MFA Rules not processed for user `872dbb91c9e54f13b1a31fc8242126df`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:10:37.816263 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-58cef75b-7e39-45b0-b4b0-4e483bbd43a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:10:37.858311 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:10:37.895391 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 692/1395] 10.222.0.26 () {58 vars in 1059 bytes} [Thu Jul 14 09:10:37 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:10:37.902878 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c78ebe-3dd1-498b-ab95-6b1bc3e56a41 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:10:37.904533 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-04c78ebe-3dd1-498b-ab95-6b1bc3e56a41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:10:37.918771 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:10:38.014795 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c78ebe-3dd1-498b-ab95-6b1bc3e56a41 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:10:38.095230 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d51d57b-f1cb-41e2-8bb7-cda37958046d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '872dbb91c9e54f13b1a31fc8242126df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:10:38.095915 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d51d57b-f1cb-41e2-8bb7-cda37958046d service glance] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:10:38.096168 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d51d57b-f1cb-41e2-8bb7-cda37958046d service glance] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:10:38.096475 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d51d57b-f1cb-41e2-8bb7-cda37958046d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:10:38.098140 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-7d51d57b-f1cb-41e2-8bb7-cda37958046d service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:10:38.167810 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:10:38.211373 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d51d57b-f1cb-41e2-8bb7-cda37958046d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:10:38.212974 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d51d57b-f1cb-41e2-8bb7-cda37958046d service glance] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:10:38.231588 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 704/1396] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:10:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:10:38.349147 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc60f22-20fc-4d09-ae97-f223d78a34db None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:10:38.349536 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc60f22-20fc-4d09-ae97-f223d78a34db None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:10:38.349884 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc60f22-20fc-4d09-ae97-f223d78a34db None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:10:38.351398 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 693/1397] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:10:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:10:38.356914 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dd2b5b-5c43-44e9-be5d-5943b95f2bbd None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:10:38.357209 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dd2b5b-5c43-44e9-be5d-5943b95f2bbd None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:10:38.357521 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dd2b5b-5c43-44e9-be5d-5943b95f2bbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:10:38.428645 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-73dd2b5b-5c43-44e9-be5d-5943b95f2bbd None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:10:38.429974 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-73dd2b5b-5c43-44e9-be5d-5943b95f2bbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:10:38.537320 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 705/1398] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:10:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:10:38.582692 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc60f22-20fc-4d09-ae97-f223d78a34db None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:10:38.584981 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-7bc60f22-20fc-4d09-ae97-f223d78a34db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:10:38.736647 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc60f22-20fc-4d09-ae97-f223d78a34db None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:10:38.789978 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:10:38.863591 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b5b818-232b-40ca-ad6b-b43c8296461b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '538fff3c5a2b487eb44ae7c1a1198102', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:10:38.864252 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b5b818-232b-40ca-ad6b-b43c8296461b service nova] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:10:38.864541 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b5b818-232b-40ca-ad6b-b43c8296461b service nova] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:10:38.864838 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b5b818-232b-40ca-ad6b-b43c8296461b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:10:38.866491 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-e3b5b818-232b-40ca-ad6b-b43c8296461b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:10:38.938112 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca1545b-f548-4ebc-b588-845169d5d2ab None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:10:38.938399 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca1545b-f548-4ebc-b588-845169d5d2ab None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:10:38.938669 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca1545b-f548-4ebc-b588-845169d5d2ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:10:38.964171 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:10:38.969953 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b5b818-232b-40ca-ad6b-b43c8296461b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:10:38.971598 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b5b818-232b-40ca-ad6b-b43c8296461b service nova] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:10:38.990736 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 694/1399] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:10:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:10:39.005612 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-9ca1545b-f548-4ebc-b588-845169d5d2ab None None] MFA Rules not processed for user `7d6267ede8d64e2abe1f4058a9797fb7`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:10:39.006787 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-9ca1545b-f548-4ebc-b588-845169d5d2ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:10:39.102809 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 706/1400] 10.222.0.26 () {58 vars in 1009 bytes} [Thu Jul 14 09:10:38 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 09:10:39.150971 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acc99ab-9c72-4d4a-8a16-54c369e8fa96 None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:10:39.151410 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acc99ab-9c72-4d4a-8a16-54c369e8fa96 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:10:39.151760 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acc99ab-9c72-4d4a-8a16-54c369e8fa96 None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:10:39.153428 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 695/1401] 10.222.0.26 () {54 vars in 942 bytes} [Thu Jul 14 09:10:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:10:39.160805 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b5fa12-5b10-4ce7-964f-f60250d28894 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:10:39.161068 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b5fa12-5b10-4ce7-964f-f60250d28894 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:10:39.161377 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b5fa12-5b10-4ce7-964f-f60250d28894 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:10:39.242924 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-a3b5fa12-5b10-4ce7-964f-f60250d28894 None None] MFA Rules not processed for user `e5a43c80514b4a13af03c7f8ebcf7a5f`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:10:39.244182 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-a3b5fa12-5b10-4ce7-964f-f60250d28894 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:10:39.300141 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:10:39.357263 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 707/1402] 10.222.0.26 () {58 vars in 1039 bytes} [Thu Jul 14 09:10:39 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:10:39.366729 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acc99ab-9c72-4d4a-8a16-54c369e8fa96 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:10:39.368265 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-7acc99ab-9c72-4d4a-8a16-54c369e8fa96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:10:39.426534 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:10:39.515601 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acc99ab-9c72-4d4a-8a16-54c369e8fa96 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:10:39.627693 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668b105c-51cc-4658-b662-0584be768103 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5a43c80514b4a13af03c7f8ebcf7a5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:10:39.628411 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668b105c-51cc-4658-b662-0584be768103 service swift] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:10:39.628622 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668b105c-51cc-4658-b662-0584be768103 service swift] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:10:39.628899 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668b105c-51cc-4658-b662-0584be768103 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:10:39.630614 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-668b105c-51cc-4658-b662-0584be768103 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:10:39.671079 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:10:39.720181 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668b105c-51cc-4658-b662-0584be768103 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:10:39.721817 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668b105c-51cc-4658-b662-0584be768103 service swift] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:10:39.730192 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 696/1403] 10.222.0.26 () {60 vars in 1384 bytes} [Thu Jul 14 09:10:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 14 09:10:41.030640 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a00f9b-741b-4504-b65e-9027a23bebad None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:10:41.031237 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a00f9b-741b-4504-b65e-9027a23bebad None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:10:41.031237 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a00f9b-741b-4504-b65e-9027a23bebad None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:10:41.034526 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 708/1404] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:10:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:10:41.038010 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0e212a-bcba-4b43-b222-fbf181c6c988 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:10:41.038491 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0e212a-bcba-4b43-b222-fbf181c6c988 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:10:41.038602 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0e212a-bcba-4b43-b222-fbf181c6c988 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:10:41.107558 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-3c0e212a-bcba-4b43-b222-fbf181c6c988 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:10:41.108771 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-3c0e212a-bcba-4b43-b222-fbf181c6c988 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:10:41.212225 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 697/1405] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:10:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:10:41.255689 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a00f9b-741b-4504-b65e-9027a23bebad None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:10:41.257208 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-89a00f9b-741b-4504-b65e-9027a23bebad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:10:41.463365 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a00f9b-741b-4504-b65e-9027a23bebad None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:10:41.463365 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:10:41.561267 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf4c247-0757-4864-a3c3-4764ca44951b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '538fff3c5a2b487eb44ae7c1a1198102', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:10:41.561931 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf4c247-0757-4864-a3c3-4764ca44951b service nova] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:10:41.562212 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf4c247-0757-4864-a3c3-4764ca44951b service nova] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:10:41.562486 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf4c247-0757-4864-a3c3-4764ca44951b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:10:41.565493 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-aaf4c247-0757-4864-a3c3-4764ca44951b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:10:41.664188 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf4c247-0757-4864-a3c3-4764ca44951b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:10:41.666050 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf4c247-0757-4864-a3c3-4764ca44951b service nova] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:10:41.686407 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 709/1406] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:10:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:10:43.765661 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de86278-1bc4-4081-9f35-f36818a62874 None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:10:43.766513 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de86278-1bc4-4081-9f35-f36818a62874 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:10:43.766513 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de86278-1bc4-4081-9f35-f36818a62874 None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:10:43.768516 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 698/1407] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:10:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:10:43.773876 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b65695b-9f5c-4051-8b42-21c47cc33de8 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:10:43.774326 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b65695b-9f5c-4051-8b42-21c47cc33de8 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:10:43.774409 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b65695b-9f5c-4051-8b42-21c47cc33de8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:10:43.820929 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:10:43.850596 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-4b65695b-9f5c-4051-8b42-21c47cc33de8 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:10:43.851862 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-4b65695b-9f5c-4051-8b42-21c47cc33de8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:10:43.966047 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 710/1408] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:10:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:10:44.014570 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de86278-1bc4-4081-9f35-f36818a62874 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:10:44.016263 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-6de86278-1bc4-4081-9f35-f36818a62874 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:10:44.049416 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:10:44.178051 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de86278-1bc4-4081-9f35-f36818a62874 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:10:44.289592 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:10:44.302144 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76216533-8bfa-4992-a392-d4489c9cf905 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '538fff3c5a2b487eb44ae7c1a1198102', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:10:44.303122 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76216533-8bfa-4992-a392-d4489c9cf905 service nova] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:10:44.303501 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76216533-8bfa-4992-a392-d4489c9cf905 service nova] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:10:44.303865 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76216533-8bfa-4992-a392-d4489c9cf905 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:10:44.305548 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-76216533-8bfa-4992-a392-d4489c9cf905 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:10:44.409389 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76216533-8bfa-4992-a392-d4489c9cf905 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:10:44.410986 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76216533-8bfa-4992-a392-d4489c9cf905 service nova] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:10:44.430855 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 699/1409] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:10:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:10:53.964391 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=51 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:10:54.428069 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=49 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:11:01.904539 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3fc0c3-7a38-4f77-8385-15b38b892b8f None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:11:01.905098 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3fc0c3-7a38-4f77-8385-15b38b892b8f None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:11:01.905098 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3fc0c3-7a38-4f77-8385-15b38b892b8f None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:11:01.906413 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 711/1410] 10.222.0.26 () {54 vars in 918 bytes} [Thu Jul 14 09:11:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:11:01.911779 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7abedee-5a2c-47d4-835a-a07387d9a062 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:11:01.912203 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7abedee-5a2c-47d4-835a-a07387d9a062 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:11:01.912347 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7abedee-5a2c-47d4-835a-a07387d9a062 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:11:01.991354 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-b7abedee-5a2c-47d4-835a-a07387d9a062 None None] MFA Rules not processed for user `355684f52e074ad9bfa7fd90bcae6caf`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:11:01.992586 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-b7abedee-5a2c-47d4-835a-a07387d9a062 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:11:02.109146 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 700/1411] 10.222.0.26 () {58 vars in 1015 bytes} [Thu Jul 14 09:11:01 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:11:02.124043 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3a2bb6-2a6b-49a0-91e8-48ce879f46f3 None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:11:02.124332 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3a2bb6-2a6b-49a0-91e8-48ce879f46f3 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:11:02.124607 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3a2bb6-2a6b-49a0-91e8-48ce879f46f3 None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:11:02.126333 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 712/1412] 10.222.0.26 () {54 vars in 918 bytes} [Thu Jul 14 09:11:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:11:02.131213 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa81a71-2397-4e31-914a-2410dfdd64fa None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:11:02.131498 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa81a71-2397-4e31-914a-2410dfdd64fa None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:11:02.131774 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa81a71-2397-4e31-914a-2410dfdd64fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:11:02.208671 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:11:02.211878 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-8fa81a71-2397-4e31-914a-2410dfdd64fa None None] MFA Rules not processed for user `538fff3c5a2b487eb44ae7c1a1198102`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:11:02.213795 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-8fa81a71-2397-4e31-914a-2410dfdd64fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:11:02.468542 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 701/1413] 10.222.0.26 () {58 vars in 1015 bytes} [Thu Jul 14 09:11:02 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 338 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:11:04.934224 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667012f3-6a8c-4110-81cb-964bda732850 None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:11:04.934683 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667012f3-6a8c-4110-81cb-964bda732850 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:11:04.934796 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667012f3-6a8c-4110-81cb-964bda732850 None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:11:04.936291 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 713/1414] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 14 09:11:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:11:04.943436 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3409393-43e9-4d3f-bb1e-b9100b621b07 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:11:04.943917 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3409393-43e9-4d3f-bb1e-b9100b621b07 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:11:04.944026 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3409393-43e9-4d3f-bb1e-b9100b621b07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:11:05.022019 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-e3409393-43e9-4d3f-bb1e-b9100b621b07 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:11:05.023906 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-e3409393-43e9-4d3f-bb1e-b9100b621b07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:11:05.081832 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:11:05.126104 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 702/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 14 09:11:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:11:05.170962 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667012f3-6a8c-4110-81cb-964bda732850 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:11:05.173594 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-667012f3-6a8c-4110-81cb-964bda732850 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:11:05.355739 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667012f3-6a8c-4110-81cb-964bda732850 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:11:05.447621 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:11:05.491695 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86912a01-edfa-42a4-aba5-60eba4d4a69c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '538fff3c5a2b487eb44ae7c1a1198102', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:11:05.492377 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86912a01-edfa-42a4-aba5-60eba4d4a69c service nova] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:11:05.492661 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86912a01-edfa-42a4-aba5-60eba4d4a69c service nova] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:11:05.492933 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86912a01-edfa-42a4-aba5-60eba4d4a69c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:11:05.494680 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-86912a01-edfa-42a4-aba5-60eba4d4a69c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:11:05.597778 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86912a01-edfa-42a4-aba5-60eba4d4a69c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:11:05.599845 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86912a01-edfa-42a4-aba5-60eba4d4a69c service nova] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:11:05.618636 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 714/1416] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:11:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:11:06.540923 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4022a0f-3997-4243-8c60-d74e05dc600d None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:11:06.541519 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4022a0f-3997-4243-8c60-d74e05dc600d None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:11:06.541519 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4022a0f-3997-4243-8c60-d74e05dc600d None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:11:06.542937 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 703/1417] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 14 09:11:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:11:06.548160 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c95a5d-5b7b-44d8-b7d1-345310cb0bd5 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:11:06.548612 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c95a5d-5b7b-44d8-b7d1-345310cb0bd5 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:11:06.548734 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c95a5d-5b7b-44d8-b7d1-345310cb0bd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:11:06.638630 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-98c95a5d-5b7b-44d8-b7d1-345310cb0bd5 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:11:06.639878 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-98c95a5d-5b7b-44d8-b7d1-345310cb0bd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:11:06.668511 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:11:06.748792 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 715/1418] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 14 09:11:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:11:06.901423 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4022a0f-3997-4243-8c60-d74e05dc600d None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:11:06.903754 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-e4022a0f-3997-4243-8c60-d74e05dc600d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:11:07.063122 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4022a0f-3997-4243-8c60-d74e05dc600d None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:11:07.105205 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:11:07.179781 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20526641-f431-4a60-9a02-fd7eb9ef496c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '538fff3c5a2b487eb44ae7c1a1198102', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:11:07.180350 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20526641-f431-4a60-9a02-fd7eb9ef496c service nova] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:11:07.180628 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20526641-f431-4a60-9a02-fd7eb9ef496c service nova] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:11:07.180898 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20526641-f431-4a60-9a02-fd7eb9ef496c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:11:07.182606 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-20526641-f431-4a60-9a02-fd7eb9ef496c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:11:07.286762 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20526641-f431-4a60-9a02-fd7eb9ef496c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:11:07.289206 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20526641-f431-4a60-9a02-fd7eb9ef496c service nova] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:11:07.314188 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 704/1419] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:11:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:11:08.188023 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bacde29-e725-481b-963f-3fd45a878de5 None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:11:08.189064 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bacde29-e725-481b-963f-3fd45a878de5 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:11:08.189064 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bacde29-e725-481b-963f-3fd45a878de5 None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:11:08.191124 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 716/1420] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 14 09:11:08 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:11:08.196213 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1d5954-1197-4460-8646-6c9fdb33047a None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:11:08.196671 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1d5954-1197-4460-8646-6c9fdb33047a None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:11:08.196778 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1d5954-1197-4460-8646-6c9fdb33047a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:11:08.254362 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:11:08.277469 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-fc1d5954-1197-4460-8646-6c9fdb33047a None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:11:08.278673 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-fc1d5954-1197-4460-8646-6c9fdb33047a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:11:08.387965 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 705/1421] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 14 09:11:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:11:08.542390 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bacde29-e725-481b-963f-3fd45a878de5 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:11:08.545218 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-4bacde29-e725-481b-963f-3fd45a878de5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:11:08.701060 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bacde29-e725-481b-963f-3fd45a878de5 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:11:08.713673 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:11:08.820579 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e998732f-ed8c-4cd7-93f2-e74b2019f44b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '538fff3c5a2b487eb44ae7c1a1198102', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:11:08.821150 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e998732f-ed8c-4cd7-93f2-e74b2019f44b service nova] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:11:08.821433 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e998732f-ed8c-4cd7-93f2-e74b2019f44b service nova] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:11:08.821704 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e998732f-ed8c-4cd7-93f2-e74b2019f44b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:11:08.823514 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-e998732f-ed8c-4cd7-93f2-e74b2019f44b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:11:08.921312 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e998732f-ed8c-4cd7-93f2-e74b2019f44b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:11:08.922933 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e998732f-ed8c-4cd7-93f2-e74b2019f44b service nova] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:11:08.945885 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 717/1422] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:11:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:11:09.785340 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3bbb8f-34ef-43ec-b6a6-9d760e65b3ca None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:11:09.786200 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3bbb8f-34ef-43ec-b6a6-9d760e65b3ca None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:11:09.786200 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3bbb8f-34ef-43ec-b6a6-9d760e65b3ca None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:11:09.788343 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 706/1423] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 14 09:11:09 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:11:09.795896 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd62099d-909a-4e5c-93dd-74c48b220543 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:11:09.796341 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd62099d-909a-4e5c-93dd-74c48b220543 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:11:09.796466 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd62099d-909a-4e5c-93dd-74c48b220543 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:11:09.818296 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:11:09.881044 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-dd62099d-909a-4e5c-93dd-74c48b220543 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:11:09.882266 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-dd62099d-909a-4e5c-93dd-74c48b220543 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:11:09.982701 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 718/1424] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 14 09:11:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:11:10.133742 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3bbb8f-34ef-43ec-b6a6-9d760e65b3ca None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:11:10.135225 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-bd3bbb8f-34ef-43ec-b6a6-9d760e65b3ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:11:10.259262 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:11:10.298159 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3bbb8f-34ef-43ec-b6a6-9d760e65b3ca None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:11:10.430099 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9191f2a8-e27a-4d8e-9056-34e98cb1317c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '538fff3c5a2b487eb44ae7c1a1198102', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:11:10.430667 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9191f2a8-e27a-4d8e-9056-34e98cb1317c service nova] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:11:10.430943 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9191f2a8-e27a-4d8e-9056-34e98cb1317c service nova] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:11:10.431234 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9191f2a8-e27a-4d8e-9056-34e98cb1317c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:11:10.432862 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-9191f2a8-e27a-4d8e-9056-34e98cb1317c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:11:10.514963 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:11:10.529720 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9191f2a8-e27a-4d8e-9056-34e98cb1317c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:11:10.531499 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9191f2a8-e27a-4d8e-9056-34e98cb1317c service nova] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:11:10.552143 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 707/1425] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:11:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:11:11.405524 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72210372-5c73-409b-a2ee-ccab28e5fce3 None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:11:11.406129 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72210372-5c73-409b-a2ee-ccab28e5fce3 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:11:11.406129 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72210372-5c73-409b-a2ee-ccab28e5fce3 None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:11:11.407510 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 719/1426] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 14 09:11:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:11:11.413601 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d948365-e9dc-4d1d-a8fe-36762a12bb81 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:11:11.414348 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d948365-e9dc-4d1d-a8fe-36762a12bb81 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:11:11.414488 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d948365-e9dc-4d1d-a8fe-36762a12bb81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:11:11.491574 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-2d948365-e9dc-4d1d-a8fe-36762a12bb81 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:11:11.492791 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-2d948365-e9dc-4d1d-a8fe-36762a12bb81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:11:11.593144 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 708/1427] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 14 09:11:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:11:11.749547 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72210372-5c73-409b-a2ee-ccab28e5fce3 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:11:11.752301 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-72210372-5c73-409b-a2ee-ccab28e5fce3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:11:11.848356 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:11:11.915392 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72210372-5c73-409b-a2ee-ccab28e5fce3 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:11:12.038226 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7233382-0f98-48e8-a225-faab6b648986 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '538fff3c5a2b487eb44ae7c1a1198102', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:11:12.038804 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7233382-0f98-48e8-a225-faab6b648986 service nova] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:11:12.039080 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7233382-0f98-48e8-a225-faab6b648986 service nova] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:11:12.039390 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7233382-0f98-48e8-a225-faab6b648986 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:11:12.041011 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-a7233382-0f98-48e8-a225-faab6b648986 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:11:12.097691 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:11:12.147101 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7233382-0f98-48e8-a225-faab6b648986 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:11:12.148767 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7233382-0f98-48e8-a225-faab6b648986 service nova] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:11:12.167356 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 720/1428] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:11:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:11:12.948698 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5446bda2-023f-4a39-80da-c8ff7ea2eca0 None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:11:12.949585 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5446bda2-023f-4a39-80da-c8ff7ea2eca0 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:11:12.949585 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5446bda2-023f-4a39-80da-c8ff7ea2eca0 None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:11:12.951725 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 709/1429] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 14 09:11:12 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:11:12.960224 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0732859f-bf91-4bec-8f4a-7082628ccd32 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:11:12.960663 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0732859f-bf91-4bec-8f4a-7082628ccd32 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:11:12.960804 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0732859f-bf91-4bec-8f4a-7082628ccd32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:11:13.039785 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-0732859f-bf91-4bec-8f4a-7082628ccd32 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:11:13.040994 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-0732859f-bf91-4bec-8f4a-7082628ccd32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:11:13.145667 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 721/1430] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 14 09:11:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:11:13.298721 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5446bda2-023f-4a39-80da-c8ff7ea2eca0 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:11:13.301214 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-5446bda2-023f-4a39-80da-c8ff7ea2eca0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:11:13.357679 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:11:13.458335 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5446bda2-023f-4a39-80da-c8ff7ea2eca0 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:11:13.581970 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e2336c-aa05-4155-9f08-2b86a3cb2074 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '538fff3c5a2b487eb44ae7c1a1198102', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:11:13.582623 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e2336c-aa05-4155-9f08-2b86a3cb2074 service nova] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:11:13.582912 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e2336c-aa05-4155-9f08-2b86a3cb2074 service nova] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:11:13.583211 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e2336c-aa05-4155-9f08-2b86a3cb2074 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:11:13.584863 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-32e2336c-aa05-4155-9f08-2b86a3cb2074 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:11:13.592035 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:11:13.677374 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e2336c-aa05-4155-9f08-2b86a3cb2074 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:11:13.678982 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e2336c-aa05-4155-9f08-2b86a3cb2074 service nova] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:11:13.697500 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 710/1431] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:11:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:11:14.550817 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b48ce53-bc82-4b6a-b2bc-5b2d319a31ee None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:11:14.551467 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b48ce53-bc82-4b6a-b2bc-5b2d319a31ee None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:11:14.551467 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b48ce53-bc82-4b6a-b2bc-5b2d319a31ee None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:11:14.553265 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 722/1432] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 14 09:11:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:11:14.560387 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e1d275-ee62-4b0f-a0ab-c7094b9f6aea None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:11:14.560892 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e1d275-ee62-4b0f-a0ab-c7094b9f6aea None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:11:14.560997 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e1d275-ee62-4b0f-a0ab-c7094b9f6aea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:11:14.642628 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-f2e1d275-ee62-4b0f-a0ab-c7094b9f6aea None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:11:14.643882 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-f2e1d275-ee62-4b0f-a0ab-c7094b9f6aea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:11:14.721439 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:11:14.753730 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 711/1433] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 14 09:11:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 09:11:14.910352 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b48ce53-bc82-4b6a-b2bc-5b2d319a31ee None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:11:14.912753 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-4b48ce53-bc82-4b6a-b2bc-5b2d319a31ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:11:14.934406 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:11:15.070622 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b48ce53-bc82-4b6a-b2bc-5b2d319a31ee None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:11:15.164516 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:11:15.192444 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336f10a3-6765-4e46-88db-01ca83f6c5ba service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '538fff3c5a2b487eb44ae7c1a1198102', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:11:15.193104 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336f10a3-6765-4e46-88db-01ca83f6c5ba service nova] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:11:15.193388 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336f10a3-6765-4e46-88db-01ca83f6c5ba service nova] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:11:15.193668 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336f10a3-6765-4e46-88db-01ca83f6c5ba service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:11:15.195425 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-336f10a3-6765-4e46-88db-01ca83f6c5ba service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:11:15.298425 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336f10a3-6765-4e46-88db-01ca83f6c5ba service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:11:15.300079 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336f10a3-6765-4e46-88db-01ca83f6c5ba service nova] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:11:15.318145 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 723/1434] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:11:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:11:16.121603 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c031e77-2c69-4997-8aef-8af4434f736b None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:11:16.122147 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c031e77-2c69-4997-8aef-8af4434f736b None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:11:16.122534 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c031e77-2c69-4997-8aef-8af4434f736b None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:11:16.124326 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 712/1435] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 14 09:11:16 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:11:16.132061 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa49b1c9-98f8-4ea0-9620-3a9cc59d89e8 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:11:16.132536 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa49b1c9-98f8-4ea0-9620-3a9cc59d89e8 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:11:16.132643 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa49b1c9-98f8-4ea0-9620-3a9cc59d89e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:11:16.223043 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-aa49b1c9-98f8-4ea0-9620-3a9cc59d89e8 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:11:16.224293 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-aa49b1c9-98f8-4ea0-9620-3a9cc59d89e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:11:16.271401 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:11:16.334298 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 724/1436] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 14 09:11:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:11:16.487257 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c031e77-2c69-4997-8aef-8af4434f736b None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:11:16.489687 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-7c031e77-2c69-4997-8aef-8af4434f736b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:11:16.657629 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c031e77-2c69-4997-8aef-8af4434f736b None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:11:16.723389 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:11:16.790349 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619040bb-564d-4da6-8bc5-7d6adfd4bb2a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '538fff3c5a2b487eb44ae7c1a1198102', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:11:16.791004 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619040bb-564d-4da6-8bc5-7d6adfd4bb2a service nova] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:11:16.791319 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619040bb-564d-4da6-8bc5-7d6adfd4bb2a service nova] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:11:16.791595 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619040bb-564d-4da6-8bc5-7d6adfd4bb2a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:11:16.793318 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-619040bb-564d-4da6-8bc5-7d6adfd4bb2a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:11:16.891290 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619040bb-564d-4da6-8bc5-7d6adfd4bb2a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:11:16.892806 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619040bb-564d-4da6-8bc5-7d6adfd4bb2a service nova] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:11:16.911113 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 713/1437] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:11:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:11:17.711527 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940706c4-7a4e-47ac-bf46-9dd5415d913f None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:11:17.712445 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940706c4-7a4e-47ac-bf46-9dd5415d913f None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:11:17.712445 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940706c4-7a4e-47ac-bf46-9dd5415d913f None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:11:17.714527 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 725/1438] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 14 09:11:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:11:17.722872 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694af81f-ba90-422c-8ff0-4256534dccab None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:11:17.723327 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694af81f-ba90-422c-8ff0-4256534dccab None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:11:17.723483 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694af81f-ba90-422c-8ff0-4256534dccab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:11:17.805896 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-694af81f-ba90-422c-8ff0-4256534dccab None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:11:17.807019 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-694af81f-ba90-422c-8ff0-4256534dccab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:11:17.809246 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:11:17.913146 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 714/1439] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 14 09:11:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:11:18.063566 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940706c4-7a4e-47ac-bf46-9dd5415d913f None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:11:18.065757 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-940706c4-7a4e-47ac-bf46-9dd5415d913f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:11:18.230564 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940706c4-7a4e-47ac-bf46-9dd5415d913f None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:11:18.259237 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:11:18.350804 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81ec096-debb-4dae-906f-fc9b5be156a3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '538fff3c5a2b487eb44ae7c1a1198102', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:11:18.351429 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81ec096-debb-4dae-906f-fc9b5be156a3 service nova] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:11:18.351707 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81ec096-debb-4dae-906f-fc9b5be156a3 service nova] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:11:18.351982 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81ec096-debb-4dae-906f-fc9b5be156a3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:11:18.353613 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-e81ec096-debb-4dae-906f-fc9b5be156a3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:11:18.458008 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81ec096-debb-4dae-906f-fc9b5be156a3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:11:18.459713 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81ec096-debb-4dae-906f-fc9b5be156a3 service nova] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:11:18.488163 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 726/1440] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:11:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:11:20.398975 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edc7803-06f0-408d-a10a-9c9ea183618e None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:11:20.399616 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edc7803-06f0-408d-a10a-9c9ea183618e None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:11:20.399616 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edc7803-06f0-408d-a10a-9c9ea183618e None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:11:20.400965 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 715/1441] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 14 09:11:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:11:20.407579 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f018781-4ffa-41a8-b5e7-19fe0efdf8ac None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:11:20.408310 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f018781-4ffa-41a8-b5e7-19fe0efdf8ac None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:11:20.408501 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f018781-4ffa-41a8-b5e7-19fe0efdf8ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:11:20.454268 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:11:20.481382 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-5f018781-4ffa-41a8-b5e7-19fe0efdf8ac None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:11:20.482606 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-5f018781-4ffa-41a8-b5e7-19fe0efdf8ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:11:20.704063 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 727/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 14 09:11:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 298 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:11:20.744748 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edc7803-06f0-408d-a10a-9c9ea183618e None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:11:20.746403 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-2edc7803-06f0-408d-a10a-9c9ea183618e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:11:20.868107 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edc7803-06f0-408d-a10a-9c9ea183618e None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:11:20.901779 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:11:20.948171 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84802d75-6d6c-4f27-8ef1-7175fdb8b5a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '952911c492444ffea4b211a24b564117', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:11:20.948791 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84802d75-6d6c-4f27-8ef1-7175fdb8b5a2 service neutron] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:11:20.949076 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84802d75-6d6c-4f27-8ef1-7175fdb8b5a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:11:20.949349 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84802d75-6d6c-4f27-8ef1-7175fdb8b5a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:11:20.950994 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-84802d75-6d6c-4f27-8ef1-7175fdb8b5a2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:11:21.086321 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84802d75-6d6c-4f27-8ef1-7175fdb8b5a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:11:21.088145 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84802d75-6d6c-4f27-8ef1-7175fdb8b5a2 service neutron] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:11:21.106674 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 716/1443] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:11:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:11:23.263922 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d60b29a-6eba-4602-ac0d-568f81aaf7ee None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:11:23.264527 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d60b29a-6eba-4602-ac0d-568f81aaf7ee None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:11:23.264527 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d60b29a-6eba-4602-ac0d-568f81aaf7ee None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:11:23.265849 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 728/1444] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 14 09:11:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:11:23.272279 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dc3ea9-15f8-4da9-9aa7-096980bcc953 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:11:23.272708 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dc3ea9-15f8-4da9-9aa7-096980bcc953 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:11:23.272804 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dc3ea9-15f8-4da9-9aa7-096980bcc953 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:11:23.345473 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-33dc3ea9-15f8-4da9-9aa7-096980bcc953 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:11:23.346683 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-33dc3ea9-15f8-4da9-9aa7-096980bcc953 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:11:23.360293 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:11:23.453576 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 717/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 14 09:11:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:11:23.491517 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d60b29a-6eba-4602-ac0d-568f81aaf7ee None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:11:23.493039 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-7d60b29a-6eba-4602-ac0d-568f81aaf7ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:11:23.605554 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d60b29a-6eba-4602-ac0d-568f81aaf7ee None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:11:23.618258 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:11:23.680029 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b3e12a-7a6f-4eac-905e-93530bef65c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '952911c492444ffea4b211a24b564117', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:11:23.680672 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b3e12a-7a6f-4eac-905e-93530bef65c8 service neutron] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:11:23.680952 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b3e12a-7a6f-4eac-905e-93530bef65c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:11:23.681218 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b3e12a-7a6f-4eac-905e-93530bef65c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:11:23.682925 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-73b3e12a-7a6f-4eac-905e-93530bef65c8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:11:23.796197 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b3e12a-7a6f-4eac-905e-93530bef65c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:11:23.797630 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b3e12a-7a6f-4eac-905e-93530bef65c8 service neutron] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:11:23.816019 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 729/1446] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:11:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:11:26.113704 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca28233f-60a0-4561-9fb2-baf67824cbd2 None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:11:26.114597 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca28233f-60a0-4561-9fb2-baf67824cbd2 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:11:26.114597 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca28233f-60a0-4561-9fb2-baf67824cbd2 None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:11:26.116762 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 718/1447] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 14 09:11:26 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:11:26.125851 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3239a844-9397-4969-9fcd-494a50ec8582 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:11:26.126620 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3239a844-9397-4969-9fcd-494a50ec8582 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:11:26.127124 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3239a844-9397-4969-9fcd-494a50ec8582 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:11:26.191215 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:11:26.212184 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-3239a844-9397-4969-9fcd-494a50ec8582 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:11:26.213427 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-3239a844-9397-4969-9fcd-494a50ec8582 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:11:26.329945 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 730/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 14 09:11:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:11:26.368864 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca28233f-60a0-4561-9fb2-baf67824cbd2 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:11:26.370333 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-ca28233f-60a0-4561-9fb2-baf67824cbd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:11:26.481783 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca28233f-60a0-4561-9fb2-baf67824cbd2 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:11:26.524564 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:11:26.562818 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7eb6224-682d-4b36-8690-6e996535089c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '952911c492444ffea4b211a24b564117', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:11:26.563511 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7eb6224-682d-4b36-8690-6e996535089c service neutron] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:11:26.563798 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7eb6224-682d-4b36-8690-6e996535089c service neutron] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:11:26.564069 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7eb6224-682d-4b36-8690-6e996535089c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:11:26.565715 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-f7eb6224-682d-4b36-8690-6e996535089c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:11:26.688663 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7eb6224-682d-4b36-8690-6e996535089c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:11:26.690117 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7eb6224-682d-4b36-8690-6e996535089c service neutron] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:11:26.710522 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 719/1449] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:11:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:11:28.846011 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff3c0e4-28e6-4a11-aac1-5326b683405c None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:11:28.846535 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff3c0e4-28e6-4a11-aac1-5326b683405c None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:11:28.846535 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff3c0e4-28e6-4a11-aac1-5326b683405c None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:11:28.848650 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 731/1450] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 14 09:11:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:11:28.856915 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124c25ea-b95f-4334-9bee-22ea20245dc0 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:11:28.857385 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124c25ea-b95f-4334-9bee-22ea20245dc0 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:11:28.857500 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124c25ea-b95f-4334-9bee-22ea20245dc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:11:28.924372 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-124c25ea-b95f-4334-9bee-22ea20245dc0 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:11:28.925584 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-124c25ea-b95f-4334-9bee-22ea20245dc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:11:28.947607 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:11:29.026396 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 720/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 14 09:11:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 09:11:29.068708 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff3c0e4-28e6-4a11-aac1-5326b683405c None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:11:29.070968 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-9ff3c0e4-28e6-4a11-aac1-5326b683405c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:11:29.198028 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff3c0e4-28e6-4a11-aac1-5326b683405c None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:11:29.220320 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:11:29.276329 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d17cfcb-ef3b-4f10-9cd4-53f75af2857c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '952911c492444ffea4b211a24b564117', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:11:29.276984 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d17cfcb-ef3b-4f10-9cd4-53f75af2857c service neutron] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:11:29.277265 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d17cfcb-ef3b-4f10-9cd4-53f75af2857c service neutron] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:11:29.277540 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d17cfcb-ef3b-4f10-9cd4-53f75af2857c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:11:29.279207 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-9d17cfcb-ef3b-4f10-9cd4-53f75af2857c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:11:29.394911 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d17cfcb-ef3b-4f10-9cd4-53f75af2857c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:11:29.396394 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d17cfcb-ef3b-4f10-9cd4-53f75af2857c service neutron] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:11:29.415205 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 732/1452] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:11:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:11:31.470467 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54859e3-3e91-4fc0-80fe-fc80b581107c None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:11:31.471345 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54859e3-3e91-4fc0-80fe-fc80b581107c None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:11:31.471345 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54859e3-3e91-4fc0-80fe-fc80b581107c None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:11:31.473529 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 721/1453] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 14 09:11:31 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:11:31.481685 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8cb27d-c165-4907-b0c5-5d357043d395 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:11:31.482113 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8cb27d-c165-4907-b0c5-5d357043d395 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:11:31.482267 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8cb27d-c165-4907-b0c5-5d357043d395 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:11:31.545880 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:11:31.550533 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-8b8cb27d-c165-4907-b0c5-5d357043d395 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:11:31.551784 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-8b8cb27d-c165-4907-b0c5-5d357043d395 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:11:31.655038 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 733/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 14 09:11:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:11:31.696916 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54859e3-3e91-4fc0-80fe-fc80b581107c None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:11:31.701002 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-d54859e3-3e91-4fc0-80fe-fc80b581107c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:11:31.825073 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54859e3-3e91-4fc0-80fe-fc80b581107c None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:11:31.875973 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:11:31.904306 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474126d9-beaa-4686-b4a6-d508cb56361d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '952911c492444ffea4b211a24b564117', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:11:31.904961 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474126d9-beaa-4686-b4a6-d508cb56361d service neutron] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:11:31.905245 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474126d9-beaa-4686-b4a6-d508cb56361d service neutron] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:11:31.905538 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474126d9-beaa-4686-b4a6-d508cb56361d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:11:31.907177 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-474126d9-beaa-4686-b4a6-d508cb56361d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:11:32.019505 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474126d9-beaa-4686-b4a6-d508cb56361d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:11:32.020928 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474126d9-beaa-4686-b4a6-d508cb56361d service neutron] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:11:32.040751 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 722/1455] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:11:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:11:34.140241 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8798c62-8069-4f59-8051-8af4238068cd None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:11:34.141162 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8798c62-8069-4f59-8051-8af4238068cd None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:11:34.141162 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8798c62-8069-4f59-8051-8af4238068cd None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:11:34.143381 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 734/1456] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 14 09:11:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:11:34.152460 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8114aad1-cac8-46cc-870b-6fee5a4e41e6 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:11:34.153162 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8114aad1-cac8-46cc-870b-6fee5a4e41e6 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:11:34.153455 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8114aad1-cac8-46cc-870b-6fee5a4e41e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:11:34.225612 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-8114aad1-cac8-46cc-870b-6fee5a4e41e6 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:11:34.226814 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-8114aad1-cac8-46cc-870b-6fee5a4e41e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:11:34.258209 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:11:34.339384 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 723/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 14 09:11:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:11:34.378153 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8798c62-8069-4f59-8051-8af4238068cd None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:11:34.379769 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-e8798c62-8069-4f59-8051-8af4238068cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:11:34.486226 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8798c62-8069-4f59-8051-8af4238068cd None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:11:34.515664 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:11:34.565159 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64d96fe-4cd6-4e58-b538-cdcf30e9956d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '952911c492444ffea4b211a24b564117', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:11:34.565811 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64d96fe-4cd6-4e58-b538-cdcf30e9956d service neutron] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:11:34.566085 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64d96fe-4cd6-4e58-b538-cdcf30e9956d service neutron] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:11:34.566349 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64d96fe-4cd6-4e58-b538-cdcf30e9956d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:11:34.568013 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-b64d96fe-4cd6-4e58-b538-cdcf30e9956d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:11:34.687818 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64d96fe-4cd6-4e58-b538-cdcf30e9956d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:11:34.688978 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64d96fe-4cd6-4e58-b538-cdcf30e9956d service neutron] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:11:34.708470 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 735/1458] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:11:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:11:36.772712 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e772f3e5-8c75-40ce-b75d-7d110f7f777e None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:11:36.773617 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e772f3e5-8c75-40ce-b75d-7d110f7f777e None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:11:36.773617 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e772f3e5-8c75-40ce-b75d-7d110f7f777e None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:11:36.775985 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 724/1459] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 14 09:11:36 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:11:36.784944 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ab66a9-fc67-4b82-a52b-7997f6e6c9de None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:11:36.785713 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ab66a9-fc67-4b82-a52b-7997f6e6c9de None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:11:36.785914 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ab66a9-fc67-4b82-a52b-7997f6e6c9de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:11:36.859988 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-32ab66a9-fc67-4b82-a52b-7997f6e6c9de None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:11:36.861117 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-32ab66a9-fc67-4b82-a52b-7997f6e6c9de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:11:36.869846 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:11:36.966768 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 736/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 14 09:11:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:11:37.005370 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e772f3e5-8c75-40ce-b75d-7d110f7f777e None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:11:37.007762 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-e772f3e5-8c75-40ce-b75d-7d110f7f777e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:11:37.130744 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e772f3e5-8c75-40ce-b75d-7d110f7f777e None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:11:37.199837 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:11:37.220478 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2c44ef-8934-4fd7-840d-5fbbce63636b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '952911c492444ffea4b211a24b564117', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:11:37.221174 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2c44ef-8934-4fd7-840d-5fbbce63636b service neutron] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:11:37.221456 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2c44ef-8934-4fd7-840d-5fbbce63636b service neutron] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:11:37.221735 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2c44ef-8934-4fd7-840d-5fbbce63636b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:11:37.223412 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-4c2c44ef-8934-4fd7-840d-5fbbce63636b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:11:37.333341 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2c44ef-8934-4fd7-840d-5fbbce63636b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:11:37.334784 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2c44ef-8934-4fd7-840d-5fbbce63636b service neutron] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:11:37.355333 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 725/1461] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:11:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:11:39.408874 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc31c39-125f-41bf-a6be-d6ba8edc2cb8 None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:11:39.409530 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc31c39-125f-41bf-a6be-d6ba8edc2cb8 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:11:39.409530 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc31c39-125f-41bf-a6be-d6ba8edc2cb8 None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:11:39.410795 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 737/1462] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 14 09:11:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:11:39.417233 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f88132-4712-48d0-9d50-be5a3ef46df1 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:11:39.417706 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f88132-4712-48d0-9d50-be5a3ef46df1 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:11:39.418388 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f88132-4712-48d0-9d50-be5a3ef46df1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:11:39.500958 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-38f88132-4712-48d0-9d50-be5a3ef46df1 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:11:39.502179 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-38f88132-4712-48d0-9d50-be5a3ef46df1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:11:39.543421 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:11:39.604968 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 726/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 14 09:11:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:11:39.646218 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc31c39-125f-41bf-a6be-d6ba8edc2cb8 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:11:39.648755 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-dcc31c39-125f-41bf-a6be-d6ba8edc2cb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:11:39.766744 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc31c39-125f-41bf-a6be-d6ba8edc2cb8 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:11:39.821966 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:11:39.860176 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258952e1-a783-4915-8f02-d05823751265 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '952911c492444ffea4b211a24b564117', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:11:39.860849 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258952e1-a783-4915-8f02-d05823751265 service neutron] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:11:39.861133 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258952e1-a783-4915-8f02-d05823751265 service neutron] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:11:39.861404 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258952e1-a783-4915-8f02-d05823751265 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:11:39.863034 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-258952e1-a783-4915-8f02-d05823751265 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:11:39.980059 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258952e1-a783-4915-8f02-d05823751265 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:11:39.981506 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258952e1-a783-4915-8f02-d05823751265 service neutron] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:11:40.000957 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 738/1464] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:11:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:11:42.054737 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84acba9b-8cef-42e7-833b-a65ea37343f7 None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:11:42.055377 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84acba9b-8cef-42e7-833b-a65ea37343f7 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:11:42.055377 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84acba9b-8cef-42e7-833b-a65ea37343f7 None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:11:42.056741 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 727/1465] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 14 09:11:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:11:42.063910 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ad9c0d-ae6a-4f45-9783-e4f889371c72 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:11:42.064669 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ad9c0d-ae6a-4f45-9783-e4f889371c72 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:11:42.064869 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ad9c0d-ae6a-4f45-9783-e4f889371c72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:11:42.137491 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-b1ad9c0d-ae6a-4f45-9783-e4f889371c72 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:11:42.138691 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-b1ad9c0d-ae6a-4f45-9783-e4f889371c72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:11:42.157344 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:11:42.242786 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 739/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 14 09:11:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:11:42.280995 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84acba9b-8cef-42e7-833b-a65ea37343f7 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:11:42.283452 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-84acba9b-8cef-42e7-833b-a65ea37343f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:11:42.410043 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84acba9b-8cef-42e7-833b-a65ea37343f7 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:11:42.482200 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:11:42.493190 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ac7e44-bab9-4b4d-86c9-a38b66a5efc5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '952911c492444ffea4b211a24b564117', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:11:42.494373 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ac7e44-bab9-4b4d-86c9-a38b66a5efc5 service neutron] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:11:42.494830 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ac7e44-bab9-4b4d-86c9-a38b66a5efc5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:11:42.495314 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ac7e44-bab9-4b4d-86c9-a38b66a5efc5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:11:42.497399 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-24ac7e44-bab9-4b4d-86c9-a38b66a5efc5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:11:42.621579 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ac7e44-bab9-4b4d-86c9-a38b66a5efc5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:11:42.623020 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ac7e44-bab9-4b4d-86c9-a38b66a5efc5 service neutron] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:11:42.642429 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 728/1467] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:11:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:11:44.673949 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb67e4c3-022f-4a18-a11a-073bfd0f7ae6 None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:11:44.674826 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb67e4c3-022f-4a18-a11a-073bfd0f7ae6 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:11:44.674826 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb67e4c3-022f-4a18-a11a-073bfd0f7ae6 None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:11:44.677069 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 740/1468] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 14 09:11:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:11:44.685238 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2991166f-c6dc-488f-810b-c834b88f664e None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:11:44.685673 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2991166f-c6dc-488f-810b-c834b88f664e None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:11:44.685786 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2991166f-c6dc-488f-810b-c834b88f664e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:11:44.752794 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-2991166f-c6dc-488f-810b-c834b88f664e None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:11:44.754379 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-2991166f-c6dc-488f-810b-c834b88f664e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:11:44.805222 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:11:44.861145 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 729/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 14 09:11:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:11:44.900762 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb67e4c3-022f-4a18-a11a-073bfd0f7ae6 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:11:44.903250 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-bb67e4c3-022f-4a18-a11a-073bfd0f7ae6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:11:45.022644 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb67e4c3-022f-4a18-a11a-073bfd0f7ae6 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:11:45.073738 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:11:45.099827 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b17b902-64ff-4310-820b-10c1ca8af8a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '952911c492444ffea4b211a24b564117', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:11:45.100473 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b17b902-64ff-4310-820b-10c1ca8af8a0 service neutron] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:11:45.100756 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b17b902-64ff-4310-820b-10c1ca8af8a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:11:45.101031 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b17b902-64ff-4310-820b-10c1ca8af8a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:11:45.102665 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-7b17b902-64ff-4310-820b-10c1ca8af8a0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:11:45.227080 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b17b902-64ff-4310-820b-10c1ca8af8a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:11:45.228554 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b17b902-64ff-4310-820b-10c1ca8af8a0 service neutron] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:11:45.248699 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 741/1470] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:11:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:11:54.858898 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=22 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:11:55.246172 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=66 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:12:04.691760 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d921c20f-e807-454a-981b-e6b1219d96d9 None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:12:04.692703 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d921c20f-e807-454a-981b-e6b1219d96d9 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:12:04.692703 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d921c20f-e807-454a-981b-e6b1219d96d9 None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:12:04.694705 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 730/1471] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:12:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:12:04.703552 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd9ba40-f996-4d20-a21e-3859cb9a7632 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:12:04.703978 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd9ba40-f996-4d20-a21e-3859cb9a7632 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:12:04.704203 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd9ba40-f996-4d20-a21e-3859cb9a7632 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:12:04.781167 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-cfd9ba40-f996-4d20-a21e-3859cb9a7632 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:12:04.782388 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-cfd9ba40-f996-4d20-a21e-3859cb9a7632 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:12:04.886113 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 742/1472] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:12:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:12:04.975913 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acecc98a-5aa8-4a28-bf43-792e6310aeac None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:12:04.976220 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acecc98a-5aa8-4a28-bf43-792e6310aeac None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:12:04.976489 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acecc98a-5aa8-4a28-bf43-792e6310aeac None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:12:04.977896 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 731/1473] 10.222.0.26 () {54 vars in 962 bytes} [Thu Jul 14 09:12:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:12:04.982768 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d24acff-7244-47d3-90dd-1dfcdbff0ba9 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:12:04.983062 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d24acff-7244-47d3-90dd-1dfcdbff0ba9 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:12:04.983392 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d24acff-7244-47d3-90dd-1dfcdbff0ba9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:12:05.065614 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-6d24acff-7244-47d3-90dd-1dfcdbff0ba9 None None] MFA Rules not processed for user `872dbb91c9e54f13b1a31fc8242126df`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:12:05.066820 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-6d24acff-7244-47d3-90dd-1dfcdbff0ba9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:12:05.090016 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:12:05.149878 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 743/1474] 10.222.0.26 () {58 vars in 1059 bytes} [Thu Jul 14 09:12:04 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:12:05.157534 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acecc98a-5aa8-4a28-bf43-792e6310aeac None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:12:05.159051 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-acecc98a-5aa8-4a28-bf43-792e6310aeac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:12:05.266208 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acecc98a-5aa8-4a28-bf43-792e6310aeac None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:12:05.346456 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddec8e1-44ad-4861-a33c-f870ae51e82f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '872dbb91c9e54f13b1a31fc8242126df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:12:05.347095 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddec8e1-44ad-4861-a33c-f870ae51e82f service glance] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:12:05.347449 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddec8e1-44ad-4861-a33c-f870ae51e82f service glance] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:12:05.347752 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddec8e1-44ad-4861-a33c-f870ae51e82f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:12:05.349401 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-6ddec8e1-44ad-4861-a33c-f870ae51e82f service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:12:05.407379 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:12:05.464421 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddec8e1-44ad-4861-a33c-f870ae51e82f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:12:05.465868 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddec8e1-44ad-4861-a33c-f870ae51e82f service glance] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:12:05.485115 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 732/1475] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:12:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:12:07.590862 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16f6190-1535-4267-a448-f3bb2e22bb03 None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:12:07.591424 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16f6190-1535-4267-a448-f3bb2e22bb03 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:12:07.591424 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16f6190-1535-4267-a448-f3bb2e22bb03 None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:12:07.592895 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 744/1476] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:12:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:12:07.601391 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d50391e-62eb-44e9-a038-11e86688a5cf None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:12:07.601842 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d50391e-62eb-44e9-a038-11e86688a5cf None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:12:07.601961 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d50391e-62eb-44e9-a038-11e86688a5cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:12:07.673677 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-0d50391e-62eb-44e9-a038-11e86688a5cf None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:12:07.674918 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-0d50391e-62eb-44e9-a038-11e86688a5cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:12:07.783679 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 733/1477] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:12:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:12:07.807416 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16f6190-1535-4267-a448-f3bb2e22bb03 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:12:07.808888 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-e16f6190-1535-4267-a448-f3bb2e22bb03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:12:07.952657 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16f6190-1535-4267-a448-f3bb2e22bb03 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:12:08.006071 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:12:08.063327 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c1065d-7ba8-41df-afb8-febed4430dd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:12:08.063946 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c1065d-7ba8-41df-afb8-febed4430dd4 admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:12:08.064224 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c1065d-7ba8-41df-afb8-febed4430dd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:12:08.064505 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c1065d-7ba8-41df-afb8-febed4430dd4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:12:08.065238 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c1065d-7ba8-41df-afb8-febed4430dd4 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:12:08.066991 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: WARNING py.warnings [None req-22c1065d-7ba8-41df-afb8-febed4430dd4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 14 09:12:08.066991 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: warnings.warn(msg) Jul 14 09:12:08.066991 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: Jul 14 09:12:08.068156 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c1065d-7ba8-41df-afb8-febed4430dd4 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:12:08.087357 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 745/1478] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jul 14 09:12:07 2022] GET /identity/v3/projects => generated 2553 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 09:12:10.103324 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29215f66-21dd-4b8b-95e0-e89500b223e6 None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:12:10.104262 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29215f66-21dd-4b8b-95e0-e89500b223e6 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:12:10.104262 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29215f66-21dd-4b8b-95e0-e89500b223e6 None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:12:10.106560 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 734/1479] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:12:10 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:12:10.112212 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642408f6-c531-479e-92e8-ffc2aa62c173 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:12:10.112672 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642408f6-c531-479e-92e8-ffc2aa62c173 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:12:10.112774 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642408f6-c531-479e-92e8-ffc2aa62c173 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:12:10.179617 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-642408f6-c531-479e-92e8-ffc2aa62c173 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:12:10.180825 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-642408f6-c531-479e-92e8-ffc2aa62c173 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:12:10.282728 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 746/1480] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:12:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:12:10.331633 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29215f66-21dd-4b8b-95e0-e89500b223e6 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:12:10.333147 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-29215f66-21dd-4b8b-95e0-e89500b223e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:12:10.343396 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:12:10.490690 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29215f66-21dd-4b8b-95e0-e89500b223e6 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:12:10.578690 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:12:10.616890 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b77aee-ad05-4ae4-994f-c83948315a0e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '538fff3c5a2b487eb44ae7c1a1198102', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:12:10.617812 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b77aee-ad05-4ae4-994f-c83948315a0e service nova] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:12:10.618175 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b77aee-ad05-4ae4-994f-c83948315a0e service nova] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:12:10.618469 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b77aee-ad05-4ae4-994f-c83948315a0e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:12:10.621050 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-79b77aee-ad05-4ae4-994f-c83948315a0e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:12:10.727669 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b77aee-ad05-4ae4-994f-c83948315a0e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:12:10.729287 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b77aee-ad05-4ae4-994f-c83948315a0e service nova] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:12:10.747969 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 735/1481] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:12:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:12:12.944246 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c565ad4-73b3-43ae-a7c0-5db9e8a86e24 None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:12:12.944783 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c565ad4-73b3-43ae-a7c0-5db9e8a86e24 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:12:12.944783 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c565ad4-73b3-43ae-a7c0-5db9e8a86e24 None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:12:12.946103 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 747/1482] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:12:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:12:12.952674 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfb0705-7e75-46df-b3df-36e11142ed27 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:12:12.953478 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfb0705-7e75-46df-b3df-36e11142ed27 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:12:12.953644 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfb0705-7e75-46df-b3df-36e11142ed27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:12:13.024722 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-3bfb0705-7e75-46df-b3df-36e11142ed27 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:12:13.025930 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-3bfb0705-7e75-46df-b3df-36e11142ed27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:12:13.073959 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:12:13.130494 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 736/1483] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:12:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:12:13.169990 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c565ad4-73b3-43ae-a7c0-5db9e8a86e24 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:12:13.172386 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-0c565ad4-73b3-43ae-a7c0-5db9e8a86e24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:12:13.201765 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:12:13.284827 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c565ad4-73b3-43ae-a7c0-5db9e8a86e24 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:12:13.361922 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6567214-3058-4f30-8187-9e8c1b181ef5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '952911c492444ffea4b211a24b564117', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:12:13.362570 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6567214-3058-4f30-8187-9e8c1b181ef5 service neutron] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:12:13.362853 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6567214-3058-4f30-8187-9e8c1b181ef5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:12:13.363128 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6567214-3058-4f30-8187-9e8c1b181ef5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:12:13.364800 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-d6567214-3058-4f30-8187-9e8c1b181ef5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:12:13.452284 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:12:13.491041 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6567214-3058-4f30-8187-9e8c1b181ef5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:12:13.492528 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6567214-3058-4f30-8187-9e8c1b181ef5 service neutron] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:12:13.511852 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 748/1484] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:12:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:12:15.544283 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d7f842-c3aa-42cd-9ec1-30bd5e93b353 None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:12:15.544938 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d7f842-c3aa-42cd-9ec1-30bd5e93b353 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:12:15.544938 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d7f842-c3aa-42cd-9ec1-30bd5e93b353 None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:12:15.546295 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 737/1485] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:12:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:12:15.551793 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19675ed9-1c5f-4754-83eb-6b7957dc38bc None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:12:15.552266 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19675ed9-1c5f-4754-83eb-6b7957dc38bc None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:12:15.552809 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19675ed9-1c5f-4754-83eb-6b7957dc38bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:12:15.623036 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-19675ed9-1c5f-4754-83eb-6b7957dc38bc None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:12:15.624270 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-19675ed9-1c5f-4754-83eb-6b7957dc38bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:12:15.738239 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 749/1486] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:12:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:12:15.775861 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d7f842-c3aa-42cd-9ec1-30bd5e93b353 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:12:15.777376 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-c2d7f842-c3aa-42cd-9ec1-30bd5e93b353 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:12:15.891295 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d7f842-c3aa-42cd-9ec1-30bd5e93b353 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:12:15.966404 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:12:15.968954 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78a17ae-5879-48e6-b5b7-9683cb963a66 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '952911c492444ffea4b211a24b564117', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:12:15.969543 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78a17ae-5879-48e6-b5b7-9683cb963a66 service neutron] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:12:15.969862 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78a17ae-5879-48e6-b5b7-9683cb963a66 service neutron] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:12:15.970108 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78a17ae-5879-48e6-b5b7-9683cb963a66 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:12:15.971840 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-b78a17ae-5879-48e6-b5b7-9683cb963a66 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:12:16.090320 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78a17ae-5879-48e6-b5b7-9683cb963a66 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:12:16.091780 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78a17ae-5879-48e6-b5b7-9683cb963a66 service neutron] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:12:16.110615 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 738/1487] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:12:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 09:12:18.208052 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5e7133-9d6d-4a30-9491-bf8aed78a912 None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:12:18.208604 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5e7133-9d6d-4a30-9491-bf8aed78a912 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:12:18.208604 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5e7133-9d6d-4a30-9491-bf8aed78a912 None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:12:18.209954 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 750/1488] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:12:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:12:18.215890 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68819017-6162-43dc-9344-00b37b101f7c None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:12:18.216336 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68819017-6162-43dc-9344-00b37b101f7c None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:12:18.216426 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68819017-6162-43dc-9344-00b37b101f7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:12:18.282806 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-68819017-6162-43dc-9344-00b37b101f7c None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:12:18.284057 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-68819017-6162-43dc-9344-00b37b101f7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:12:18.352117 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:12:18.399210 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 739/1489] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:12:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:12:18.440671 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5e7133-9d6d-4a30-9491-bf8aed78a912 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:12:18.442923 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-3f5e7133-9d6d-4a30-9491-bf8aed78a912 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:12:18.488242 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:12:18.582346 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5e7133-9d6d-4a30-9491-bf8aed78a912 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:12:18.658423 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf204f9-29ab-4b06-85ea-e54e361c2b91 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '952911c492444ffea4b211a24b564117', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:12:18.659073 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf204f9-29ab-4b06-85ea-e54e361c2b91 service neutron] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:12:18.659390 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf204f9-29ab-4b06-85ea-e54e361c2b91 service neutron] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:12:18.659663 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf204f9-29ab-4b06-85ea-e54e361c2b91 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:12:18.661324 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-7cf204f9-29ab-4b06-85ea-e54e361c2b91 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:12:18.751717 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:12:18.782962 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf204f9-29ab-4b06-85ea-e54e361c2b91 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:12:18.784455 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf204f9-29ab-4b06-85ea-e54e361c2b91 service neutron] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:12:18.804299 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 751/1490] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:12:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:12:21.006492 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3109f2db-bc3b-4b80-9586-ab829967a31d None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:12:21.007069 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3109f2db-bc3b-4b80-9586-ab829967a31d None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:12:21.007069 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3109f2db-bc3b-4b80-9586-ab829967a31d None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:12:21.008471 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 740/1491] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jul 14 09:12:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:12:21.014555 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbca72f9-d824-47bf-93da-020bdf36d0f4 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:12:21.015009 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbca72f9-d824-47bf-93da-020bdf36d0f4 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:12:21.015115 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbca72f9-d824-47bf-93da-020bdf36d0f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:12:21.091232 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-bbca72f9-d824-47bf-93da-020bdf36d0f4 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:12:21.092486 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-bbca72f9-d824-47bf-93da-020bdf36d0f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:12:21.194037 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 752/1492] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jul 14 09:12:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:12:21.241067 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3109f2db-bc3b-4b80-9586-ab829967a31d None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:12:21.242694 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-3109f2db-bc3b-4b80-9586-ab829967a31d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:12:21.357922 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3109f2db-bc3b-4b80-9586-ab829967a31d None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:12:21.437846 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fed153-aed0-4a01-ab89-1ca6c5c65610 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '952911c492444ffea4b211a24b564117', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:12:21.438479 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fed153-aed0-4a01-ab89-1ca6c5c65610 service neutron] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:12:21.438746 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fed153-aed0-4a01-ab89-1ca6c5c65610 service neutron] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:12:21.439030 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fed153-aed0-4a01-ab89-1ca6c5c65610 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:12:21.440792 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-e5fed153-aed0-4a01-ab89-1ca6c5c65610 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:12:21.450804 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:12:21.569897 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fed153-aed0-4a01-ab89-1ca6c5c65610 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:12:21.571413 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fed153-aed0-4a01-ab89-1ca6c5c65610 service neutron] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:12:21.590119 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 741/1493] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:12:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:12:31.192120 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=82 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:12:31.587588 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=50 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:13:55.079768 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5722773c-1b0c-46cc-bf01-7444deaef1cc None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:13:55.080451 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5722773c-1b0c-46cc-bf01-7444deaef1cc None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:13:55.080451 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5722773c-1b0c-46cc-bf01-7444deaef1cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:13:55.167097 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-5722773c-1b0c-46cc-bf01-7444deaef1cc None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:13:55.168338 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-5722773c-1b0c-46cc-bf01-7444deaef1cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:13:55.283718 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 753/1494] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:13:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:13:55.292275 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a17facf-fed6-4f1a-8061-1ad06f96899d None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:13:55.292828 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a17facf-fed6-4f1a-8061-1ad06f96899d None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:13:55.292828 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a17facf-fed6-4f1a-8061-1ad06f96899d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:13:55.373340 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-2a17facf-fed6-4f1a-8061-1ad06f96899d None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:13:55.374504 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-2a17facf-fed6-4f1a-8061-1ad06f96899d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:13:55.482541 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 742/1495] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:13:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:13:55.488053 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5722773c-1b0c-46cc-bf01-7444deaef1cc None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:13:55.490047 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-5722773c-1b0c-46cc-bf01-7444deaef1cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:13:55.508687 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5722773c-1b0c-46cc-bf01-7444deaef1cc None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:13:55.583766 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:13:55.619781 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1995eb-9d87-4026-a6c1-5523de760fab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:13:55.620443 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1995eb-9d87-4026-a6c1-5523de760fab admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:13:55.620727 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1995eb-9d87-4026-a6c1-5523de760fab admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:13:55.620995 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1995eb-9d87-4026-a6c1-5523de760fab admin admin] PATH_INFO: `/v3/domains` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:13:55.621723 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1995eb-9d87-4026-a6c1-5523de760fab admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:13:55.622018 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1995eb-9d87-4026-a6c1-5523de760fab admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:13:55.623037 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: WARNING py.warnings [None req-2a1995eb-9d87-4026-a6c1-5523de760fab admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 14 09:13:55.623037 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: warnings.warn(msg) Jul 14 09:13:55.623037 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: Jul 14 09:13:55.624212 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1995eb-9d87-4026-a6c1-5523de760fab admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:13:55.636210 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 754/1496] 10.222.0.26 () {58 vars in 1152 bytes} [Thu Jul 14 09:13:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:13:55.644644 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a17facf-fed6-4f1a-8061-1ad06f96899d None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:13:55.654508 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a17facf-fed6-4f1a-8061-1ad06f96899d None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:13:55.717618 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:13:55.756198 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2222d0a5-c431-4756-8ddd-85afd6b12087 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:13:55.757288 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2222d0a5-c431-4756-8ddd-85afd6b12087 admin admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:13:55.757565 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2222d0a5-c431-4756-8ddd-85afd6b12087 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:13:55.757833 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2222d0a5-c431-4756-8ddd-85afd6b12087 admin admin] PATH_INFO: `/v3/projects` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:13:55.758524 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2222d0a5-c431-4756-8ddd-85afd6b12087 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:13:55.760272 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: WARNING py.warnings [None req-2222d0a5-c431-4756-8ddd-85afd6b12087 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 14 09:13:55.760272 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: warnings.warn(msg) Jul 14 09:13:55.760272 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: Jul 14 09:13:55.761455 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2222d0a5-c431-4756-8ddd-85afd6b12087 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:13:55.821128 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 743/1497] 10.222.0.26 () {60 vars in 1152 bytes} [Thu Jul 14 09:13:55 2022] POST /identity/v3/projects => generated 379 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 09:13:55.825604 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1995eb-9d87-4026-a6c1-5523de760fab admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:13:55.827252 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-2a1995eb-9d87-4026-a6c1-5523de760fab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:13:55.861430 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1995eb-9d87-4026-a6c1-5523de760fab admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:13:55.962745 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65238e89-4295-45af-b9ba-28cb793d99d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:13:55.963380 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65238e89-4295-45af-b9ba-28cb793d99d8 admin admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:13:55.963666 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65238e89-4295-45af-b9ba-28cb793d99d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:13:55.963944 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65238e89-4295-45af-b9ba-28cb793d99d8 admin admin] PATH_INFO: `/v3/users` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:13:55.964639 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65238e89-4295-45af-b9ba-28cb793d99d8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:13:55.965666 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: WARNING py.warnings [None req-65238e89-4295-45af-b9ba-28cb793d99d8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 14 09:13:55.965666 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: warnings.warn(msg) Jul 14 09:13:55.965666 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: Jul 14 09:13:55.966887 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65238e89-4295-45af-b9ba-28cb793d99d8 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:13:56.016274 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 755/1498] 10.222.0.26 () {60 vars in 1143 bytes} [Thu Jul 14 09:13:55 2022] POST /identity/v3/users => generated 290 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 09:13:56.022053 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2222d0a5-c431-4756-8ddd-85afd6b12087 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:13:56.023617 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-2222d0a5-c431-4756-8ddd-85afd6b12087 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:13:56.089668 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2222d0a5-c431-4756-8ddd-85afd6b12087 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:13:56.203369 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:13:56.217511 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03c7a88-5313-4702-b014-a74eb96fcd85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:13:56.218376 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03c7a88-5313-4702-b014-a74eb96fcd85 admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:13:56.218737 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03c7a88-5313-4702-b014-a74eb96fcd85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:13:56.219082 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03c7a88-5313-4702-b014-a74eb96fcd85 admin admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:13:56.219923 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03c7a88-5313-4702-b014-a74eb96fcd85 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:13:56.221395 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: WARNING py.warnings [None req-d03c7a88-5313-4702-b014-a74eb96fcd85 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 14 09:13:56.221395 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: warnings.warn(msg) Jul 14 09:13:56.221395 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: Jul 14 09:13:56.222667 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03c7a88-5313-4702-b014-a74eb96fcd85 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:13:56.239223 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 744/1499] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jul 14 09:13:56 2022] GET /identity/v3/roles => generated 1420 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 09:13:56.243565 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65238e89-4295-45af-b9ba-28cb793d99d8 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:13:56.245110 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-65238e89-4295-45af-b9ba-28cb793d99d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:13:56.261916 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65238e89-4295-45af-b9ba-28cb793d99d8 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:13:56.288935 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:13:56.366717 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9487c8-97e5-4151-9b24-c1421f18e9a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:13:56.367429 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9487c8-97e5-4151-9b24-c1421f18e9a5 admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:13:56.367708 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9487c8-97e5-4151-9b24-c1421f18e9a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:13:56.367983 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9487c8-97e5-4151-9b24-c1421f18e9a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:13:56.368782 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9487c8-97e5-4151-9b24-c1421f18e9a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:13:56.369819 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: WARNING py.warnings [None req-8a9487c8-97e5-4151-9b24-c1421f18e9a5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 14 09:13:56.369819 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: warnings.warn(msg) Jul 14 09:13:56.369819 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: Jul 14 09:13:56.370707 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9487c8-97e5-4151-9b24-c1421f18e9a5 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:13:56.383712 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 756/1500] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jul 14 09:13:56 2022] GET /identity/v3/roles => generated 1420 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 09:13:56.388148 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03c7a88-5313-4702-b014-a74eb96fcd85 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:13:56.389833 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-d03c7a88-5313-4702-b014-a74eb96fcd85 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:13:56.425960 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03c7a88-5313-4702-b014-a74eb96fcd85 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:13:56.532559 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b25ee7b-7202-4fc4-a3de-219d5bd0f16a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:13:56.533530 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b25ee7b-7202-4fc4-a3de-219d5bd0f16a admin admin] REQUEST_METHOD: `PUT` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:13:56.533829 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b25ee7b-7202-4fc4-a3de-219d5bd0f16a admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:13:56.534101 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b25ee7b-7202-4fc4-a3de-219d5bd0f16a admin admin] PATH_INFO: `/v3/projects/99f6ab7c18f2475499fd7bab3c4ca1af/users/9d14322afcfa40d78965cad7a3b0f49d/roles/d1691322ce6a4edab7ce722edf0f2988` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:13:56.560698 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b25ee7b-7202-4fc4-a3de-219d5bd0f16a admin admin] RBAC: Authorizing `identity:create_grant(project_id=99f6ab7c18f2475499fd7bab3c4ca1af, user_id=9d14322afcfa40d78965cad7a3b0f49d, role_id=d1691322ce6a4edab7ce722edf0f2988)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:13:56.561795 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: WARNING py.warnings [None req-4b25ee7b-7202-4fc4-a3de-219d5bd0f16a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 14 09:13:56.561795 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: warnings.warn(msg) Jul 14 09:13:56.561795 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: Jul 14 09:13:56.563755 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b25ee7b-7202-4fc4-a3de-219d5bd0f16a admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:13:56.590758 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 745/1501] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jul 14 09:13:56 2022] PUT /identity/v3/projects/99f6ab7c18f2475499fd7bab3c4ca1af/users/9d14322afcfa40d78965cad7a3b0f49d/roles/d1691322ce6a4edab7ce722edf0f2988 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:13:56.603135 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b1a483-16e1-4595-98e3-61c73533c541 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:13:56.603570 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b1a483-16e1-4595-98e3-61c73533c541 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:13:56.603850 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b1a483-16e1-4595-98e3-61c73533c541 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:13:56.662392 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-16b1a483-16e1-4595-98e3-61c73533c541 None None] MFA Rules not processed for user `9d14322afcfa40d78965cad7a3b0f49d`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:13:56.664228 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-16b1a483-16e1-4595-98e3-61c73533c541 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:13:56.759152 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 757/1502] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:13:56 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:13:56.772422 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629bc525-494c-42f8-898c-48d5d64592e2 None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:13:56.772713 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629bc525-494c-42f8-898c-48d5d64592e2 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:13:56.772975 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629bc525-494c-42f8-898c-48d5d64592e2 None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:13:56.774357 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 746/1503] 10.222.0.26 () {54 vars in 963 bytes} [Thu Jul 14 09:13:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:13:56.779157 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e2adee-ecd2-46e3-b83d-376537d03a0c None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:13:56.779502 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e2adee-ecd2-46e3-b83d-376537d03a0c None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:13:56.779925 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e2adee-ecd2-46e3-b83d-376537d03a0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:13:56.811120 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:13:56.857208 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-b7e2adee-ecd2-46e3-b83d-376537d03a0c None None] MFA Rules not processed for user `e76f1952a5b146808844b9f2b7395035`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:13:56.858545 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-b7e2adee-ecd2-46e3-b83d-376537d03a0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:13:56.939352 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 758/1504] 10.222.0.26 () {58 vars in 1060 bytes} [Thu Jul 14 09:13:56 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:13:56.947426 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629bc525-494c-42f8-898c-48d5d64592e2 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:13:56.948947 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-629bc525-494c-42f8-898c-48d5d64592e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:13:57.004470 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:13:57.069674 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629bc525-494c-42f8-898c-48d5d64592e2 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:13:57.146174 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391add5b-b3d0-4954-903e-31a3a4197238 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e76f1952a5b146808844b9f2b7395035', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:13:57.146827 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391add5b-b3d0-4954-903e-31a3a4197238 service cinder] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:13:57.147107 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391add5b-b3d0-4954-903e-31a3a4197238 service cinder] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:13:57.147488 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391add5b-b3d0-4954-903e-31a3a4197238 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:13:57.149149 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-391add5b-b3d0-4954-903e-31a3a4197238 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:13:57.263159 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391add5b-b3d0-4954-903e-31a3a4197238 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:13:57.264690 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391add5b-b3d0-4954-903e-31a3a4197238 service cinder] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:13:57.268228 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:13:57.286708 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 747/1505] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:13:56 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:13:57.328535 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32eb0b2a-caca-4a90-a646-09293bb64ef8 None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:13:57.329020 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32eb0b2a-caca-4a90-a646-09293bb64ef8 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:13:57.329127 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32eb0b2a-caca-4a90-a646-09293bb64ef8 None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:13:57.330380 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 759/1506] 10.222.0.26 () {52 vars in 829 bytes} [Thu Jul 14 09:13:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:13:57.334868 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391add5b-b3d0-4954-903e-31a3a4197238 service cinder] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:13:57.336694 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-391add5b-b3d0-4954-903e-31a3a4197238 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:13:57.442572 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391add5b-b3d0-4954-903e-31a3a4197238 service cinder] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:13:57.548595 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9fb06f-b623-4930-b64f-ae542406a7de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:13:57.549942 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9fb06f-b623-4930-b64f-ae542406a7de admin admin] REQUEST_METHOD: `DELETE` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:13:57.550259 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9fb06f-b623-4930-b64f-ae542406a7de admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:13:57.550537 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9fb06f-b623-4930-b64f-ae542406a7de admin admin] PATH_INFO: `/v3/users/9d14322afcfa40d78965cad7a3b0f49d` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:13:57.566112 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9fb06f-b623-4930-b64f-ae542406a7de admin admin] RBAC: Authorizing `identity:delete_user(user_id=9d14322afcfa40d78965cad7a3b0f49d)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:13:57.567154 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: WARNING py.warnings [None req-2e9fb06f-b623-4930-b64f-ae542406a7de admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 14 09:13:57.567154 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: warnings.warn(msg) Jul 14 09:13:57.567154 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: Jul 14 09:13:57.568438 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9fb06f-b623-4930-b64f-ae542406a7de admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:13:57.579028 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:13:57.646954 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-2e9fb06f-b623-4930-b64f-ae542406a7de admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9d14322afcfa40d78965cad7a3b0f49d'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:13:57.657970 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-2e9fb06f-b623-4930-b64f-ae542406a7de admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9d14322afcfa40d78965cad7a3b0f49d'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:13:57.658395 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-2e9fb06f-b623-4930-b64f-ae542406a7de admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9d14322afcfa40d78965cad7a3b0f49d'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:13:57.658927 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-2e9fb06f-b623-4930-b64f-ae542406a7de admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9d14322afcfa40d78965cad7a3b0f49d'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:13:57.672662 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-2e9fb06f-b623-4930-b64f-ae542406a7de admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9d14322afcfa40d78965cad7a3b0f49d'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:13:57.705608 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 748/1507] 10.222.0.26 () {58 vars in 1223 bytes} [Thu Jul 14 09:13:57 2022] DELETE /identity/v3/users/9d14322afcfa40d78965cad7a3b0f49d => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:13:57.712397 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bd78b7-c67a-45e4-a625-e315a57bf889 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:13:57.712717 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bd78b7-c67a-45e4-a625-e315a57bf889 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:13:57.713046 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bd78b7-c67a-45e4-a625-e315a57bf889 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:13:57.779172 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-04bd78b7-c67a-45e4-a625-e315a57bf889 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:13:57.780409 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-04bd78b7-c67a-45e4-a625-e315a57bf889 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:13:57.859373 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:13:57.872566 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 760/1508] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:13:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:13:57.882659 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9fb06f-b623-4930-b64f-ae542406a7de admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:13:57.883921 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-2e9fb06f-b623-4930-b64f-ae542406a7de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:13:57.987942 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9fb06f-b623-4930-b64f-ae542406a7de admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:13:58.046051 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:13:58.068247 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfaf742-97c2-4d69-a4bd-708e06f170c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '952911c492444ffea4b211a24b564117', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:13:58.068889 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfaf742-97c2-4d69-a4bd-708e06f170c5 service neutron] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:13:58.069173 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfaf742-97c2-4d69-a4bd-708e06f170c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:13:58.069446 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfaf742-97c2-4d69-a4bd-708e06f170c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:13:58.071070 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-fdfaf742-97c2-4d69-a4bd-708e06f170c5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:13:58.198317 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfaf742-97c2-4d69-a4bd-708e06f170c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:13:58.199723 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfaf742-97c2-4d69-a4bd-708e06f170c5 service neutron] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:13:58.221081 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 749/1509] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:13:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:13:58.606348 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bd78b7-c67a-45e4-a625-e315a57bf889 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:13:58.607759 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-04bd78b7-c67a-45e4-a625-e315a57bf889 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:13:58.620043 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bd78b7-c67a-45e4-a625-e315a57bf889 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:13:58.720644 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d254d1-c1e6-4a04-9c1b-bf881f91818f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:13:58.721523 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d254d1-c1e6-4a04-9c1b-bf881f91818f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:13:58.721809 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d254d1-c1e6-4a04-9c1b-bf881f91818f admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:13:58.722086 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d254d1-c1e6-4a04-9c1b-bf881f91818f admin admin] PATH_INFO: `/v3/projects/99f6ab7c18f2475499fd7bab3c4ca1af` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:13:58.732435 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d254d1-c1e6-4a04-9c1b-bf881f91818f admin admin] RBAC: Authorizing `identity:delete_project(project_id=99f6ab7c18f2475499fd7bab3c4ca1af)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:13:58.733410 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: WARNING py.warnings [None req-d4d254d1-c1e6-4a04-9c1b-bf881f91818f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 14 09:13:58.733410 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: warnings.warn(msg) Jul 14 09:13:58.733410 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: Jul 14 09:13:58.734615 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d254d1-c1e6-4a04-9c1b-bf881f91818f admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:13:58.801340 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-d4d254d1-c1e6-4a04-9c1b-bf881f91818f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '99f6ab7c18f2475499fd7bab3c4ca1af', 'request_id': 'req-d4d254d1-c1e6-4a04-9c1b-bf881f91818f'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:13:58.814777 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-d4d254d1-c1e6-4a04-9c1b-bf881f91818f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '99f6ab7c18f2475499fd7bab3c4ca1af', 'request_id': 'req-d4d254d1-c1e6-4a04-9c1b-bf881f91818f'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:13:58.821270 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-d4d254d1-c1e6-4a04-9c1b-bf881f91818f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '99f6ab7c18f2475499fd7bab3c4ca1af', 'request_id': 'req-d4d254d1-c1e6-4a04-9c1b-bf881f91818f'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:13:58.847695 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-d4d254d1-c1e6-4a04-9c1b-bf881f91818f admin admin] The token cache is being invalidate because project 99f6ab7c18f2475499fd7bab3c4ca1af was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80382) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 09:13:58.847982 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-d4d254d1-c1e6-4a04-9c1b-bf881f91818f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:13:58.848400 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-d4d254d1-c1e6-4a04-9c1b-bf881f91818f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:13:58.849851 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 761/1510] 10.222.0.26 () {58 vars in 1232 bytes} [Thu Jul 14 09:13:58 2022] DELETE /identity/v3/projects/99f6ab7c18f2475499fd7bab3c4ca1af => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:14:08.218288 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=62 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:14:08.845664 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:15:40.385060 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5066bd5-06ae-4295-9628-c2a715fbdf3f None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:15:40.386336 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5066bd5-06ae-4295-9628-c2a715fbdf3f None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:15:40.386336 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5066bd5-06ae-4295-9628-c2a715fbdf3f None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:15:40.387699 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 750/1511] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 14 09:15:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:15:40.392706 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a17a50-813c-43dc-beb2-b59a052fcce5 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:15:40.393155 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a17a50-813c-43dc-beb2-b59a052fcce5 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:15:40.393257 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a17a50-813c-43dc-beb2-b59a052fcce5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:15:40.477798 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-08a17a50-813c-43dc-beb2-b59a052fcce5 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:15:40.479001 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-08a17a50-813c-43dc-beb2-b59a052fcce5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:15:40.594535 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 762/1512] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 14 09:15:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:15:40.750972 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5066bd5-06ae-4295-9628-c2a715fbdf3f None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:15:40.755110 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-c5066bd5-06ae-4295-9628-c2a715fbdf3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:15:40.884498 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5066bd5-06ae-4295-9628-c2a715fbdf3f None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:15:40.891591 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b606324-cd6f-4155-9bcc-ea2bd778f42c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '538fff3c5a2b487eb44ae7c1a1198102', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:15:40.892258 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b606324-cd6f-4155-9bcc-ea2bd778f42c service nova] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:15:40.892542 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b606324-cd6f-4155-9bcc-ea2bd778f42c service nova] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:15:40.892824 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b606324-cd6f-4155-9bcc-ea2bd778f42c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:15:40.901743 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b606324-cd6f-4155-9bcc-ea2bd778f42c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:15:40.903399 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b606324-cd6f-4155-9bcc-ea2bd778f42c service nova] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:15:40.913009 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 751/1513] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:15:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:15:41.731013 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ba0b7d-3a4d-40ce-bb3f-f714c7739ee5 None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:15:41.731614 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ba0b7d-3a4d-40ce-bb3f-f714c7739ee5 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:15:41.731614 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ba0b7d-3a4d-40ce-bb3f-f714c7739ee5 None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:15:41.732999 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 763/1514] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 14 09:15:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:15:41.739028 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a10a4d4-38d6-4262-95af-ffeef5038fe4 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:15:41.739792 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a10a4d4-38d6-4262-95af-ffeef5038fe4 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:15:41.739970 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a10a4d4-38d6-4262-95af-ffeef5038fe4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:15:41.768799 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-1a10a4d4-38d6-4262-95af-ffeef5038fe4 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:15:41.770024 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-1a10a4d4-38d6-4262-95af-ffeef5038fe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:15:41.778854 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 752/1515] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 14 09:15:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:15:41.928793 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ba0b7d-3a4d-40ce-bb3f-f714c7739ee5 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:15:41.947467 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ba0b7d-3a4d-40ce-bb3f-f714c7739ee5 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:15:41.954326 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fe5c43-7af8-4beb-b9f5-9e863267bfb3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '538fff3c5a2b487eb44ae7c1a1198102', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:15:41.954972 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fe5c43-7af8-4beb-b9f5-9e863267bfb3 service nova] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:15:41.955283 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fe5c43-7af8-4beb-b9f5-9e863267bfb3 service nova] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:15:41.955537 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fe5c43-7af8-4beb-b9f5-9e863267bfb3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:15:41.964972 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fe5c43-7af8-4beb-b9f5-9e863267bfb3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:15:41.966643 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fe5c43-7af8-4beb-b9f5-9e863267bfb3 service nova] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:15:41.977004 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 764/1516] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:15:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:15:41.997679 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a338b6-f74e-43f4-b29e-201b8bc7e572 None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:15:41.997967 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a338b6-f74e-43f4-b29e-201b8bc7e572 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:15:41.998248 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a338b6-f74e-43f4-b29e-201b8bc7e572 None None] PATH_INFO: `/v3` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:15:41.999777 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 753/1517] 10.222.0.17 () {54 vars in 925 bytes} [Thu Jul 14 09:15:41 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:15:42.011179 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d30897-7def-4d4b-9fb2-6c4d14709449 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:15:42.011589 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d30897-7def-4d4b-9fb2-6c4d14709449 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:15:42.011858 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d30897-7def-4d4b-9fb2-6c4d14709449 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:15:42.076521 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-d8d30897-7def-4d4b-9fb2-6c4d14709449 None None] MFA Rules not processed for user `355684f52e074ad9bfa7fd90bcae6caf`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:15:42.077748 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-d8d30897-7def-4d4b-9fb2-6c4d14709449 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:15:42.099827 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:15:42.157944 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 765/1518] 10.222.0.17 () {58 vars in 1015 bytes} [Thu Jul 14 09:15:42 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:15:50.955490 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ea33ff-4fb5-4e46-8fec-bc651172d1fb None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:15:50.956389 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ea33ff-4fb5-4e46-8fec-bc651172d1fb None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:15:50.956389 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ea33ff-4fb5-4e46-8fec-bc651172d1fb None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:15:50.958654 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 754/1519] 10.222.0.26 () {54 vars in 960 bytes} [Thu Jul 14 09:15:50 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:15:50.966635 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f115a0c7-99b5-41fd-9285-1f27b99c94df None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:15:50.967426 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f115a0c7-99b5-41fd-9285-1f27b99c94df None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:15:50.967571 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f115a0c7-99b5-41fd-9285-1f27b99c94df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:15:50.989611 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-f115a0c7-99b5-41fd-9285-1f27b99c94df None None] MFA Rules not processed for user `355684f52e074ad9bfa7fd90bcae6caf`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:15:50.991518 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-f115a0c7-99b5-41fd-9285-1f27b99c94df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:15:51.001721 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 766/1520] 10.222.0.26 () {58 vars in 1057 bytes} [Thu Jul 14 09:15:50 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:15:51.009727 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ea33ff-4fb5-4e46-8fec-bc651172d1fb None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:15:51.025576 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ea33ff-4fb5-4e46-8fec-bc651172d1fb None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:15:51.031505 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d57ba5-55ec-4c7c-94a4-d05567ffcb12 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '355684f52e074ad9bfa7fd90bcae6caf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:15:51.032145 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d57ba5-55ec-4c7c-94a4-d05567ffcb12 service placement] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:15:51.032423 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d57ba5-55ec-4c7c-94a4-d05567ffcb12 service placement] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:15:51.032718 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d57ba5-55ec-4c7c-94a4-d05567ffcb12 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:15:51.041529 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d57ba5-55ec-4c7c-94a4-d05567ffcb12 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:15:51.043124 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d57ba5-55ec-4c7c-94a4-d05567ffcb12 service placement] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:15:51.054138 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 755/1521] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:15:51 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:15:53.225053 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d5022b-704d-442b-b431-cf8691559d23 None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:15:53.225674 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d5022b-704d-442b-b431-cf8691559d23 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:15:53.225674 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d5022b-704d-442b-b431-cf8691559d23 None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:15:53.227873 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 767/1522] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 14 09:15:53 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:15:53.236188 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4f7d6c-3b6c-476d-83fb-1c748975838b None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:15:53.236622 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4f7d6c-3b6c-476d-83fb-1c748975838b None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:15:53.236767 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4f7d6c-3b6c-476d-83fb-1c748975838b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:15:53.264550 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-bd4f7d6c-3b6c-476d-83fb-1c748975838b None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:15:53.265761 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-bd4f7d6c-3b6c-476d-83fb-1c748975838b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:15:53.274382 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 756/1523] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 14 09:15:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:15:53.438314 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d5022b-704d-442b-b431-cf8691559d23 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:15:53.462220 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d5022b-704d-442b-b431-cf8691559d23 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:15:53.468696 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffb9238-8f29-4e91-9020-04fb9e3400b6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '538fff3c5a2b487eb44ae7c1a1198102', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:15:53.469459 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffb9238-8f29-4e91-9020-04fb9e3400b6 service nova] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:15:53.469675 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffb9238-8f29-4e91-9020-04fb9e3400b6 service nova] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:15:53.470008 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffb9238-8f29-4e91-9020-04fb9e3400b6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:15:53.479578 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffb9238-8f29-4e91-9020-04fb9e3400b6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:15:53.481120 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffb9238-8f29-4e91-9020-04fb9e3400b6 service nova] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:15:53.490972 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 768/1524] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:15:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:15:54.309055 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9c9ae7-0a38-4bc7-aa8f-8d64848c5ece None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:15:54.309664 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9c9ae7-0a38-4bc7-aa8f-8d64848c5ece None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:15:54.309664 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9c9ae7-0a38-4bc7-aa8f-8d64848c5ece None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:15:54.311881 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 757/1525] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 14 09:15:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:15:54.320256 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c901e470-c55a-4f3c-9849-83f1b164b745 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:15:54.321022 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c901e470-c55a-4f3c-9849-83f1b164b745 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:15:54.321185 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c901e470-c55a-4f3c-9849-83f1b164b745 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:15:54.346707 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-c901e470-c55a-4f3c-9849-83f1b164b745 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:15:54.347940 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-c901e470-c55a-4f3c-9849-83f1b164b745 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:15:54.357662 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 769/1526] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 14 09:15:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:15:54.521660 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9c9ae7-0a38-4bc7-aa8f-8d64848c5ece None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:15:54.546149 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9c9ae7-0a38-4bc7-aa8f-8d64848c5ece None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:15:54.554483 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9aa8fb1-9bfd-41f3-94b8-9b092feb685a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '538fff3c5a2b487eb44ae7c1a1198102', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:15:54.555488 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9aa8fb1-9bfd-41f3-94b8-9b092feb685a service nova] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:15:54.555863 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9aa8fb1-9bfd-41f3-94b8-9b092feb685a service nova] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:15:54.556224 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9aa8fb1-9bfd-41f3-94b8-9b092feb685a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:15:54.570777 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9aa8fb1-9bfd-41f3-94b8-9b092feb685a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:15:54.572328 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9aa8fb1-9bfd-41f3-94b8-9b092feb685a service nova] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:15:54.582340 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 758/1527] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:15:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:15:58.692780 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6ec420-c956-408c-8fef-3192637f2aad None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:15:58.693614 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6ec420-c956-408c-8fef-3192637f2aad None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:15:58.693614 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6ec420-c956-408c-8fef-3192637f2aad None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:15:58.695282 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 770/1528] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 14 09:15:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:15:58.700790 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49043aa-c8b4-42b0-b237-ab26640a6c25 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:15:58.701270 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49043aa-c8b4-42b0-b237-ab26640a6c25 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:15:58.701376 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49043aa-c8b4-42b0-b237-ab26640a6c25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:15:58.721219 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-d49043aa-c8b4-42b0-b237-ab26640a6c25 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:15:58.722426 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-d49043aa-c8b4-42b0-b237-ab26640a6c25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:15:58.730705 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 759/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 14 09:15:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:15:58.807618 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6ec420-c956-408c-8fef-3192637f2aad None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:15:58.809566 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-4d6ec420-c956-408c-8fef-3192637f2aad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:15:58.890711 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6ec420-c956-408c-8fef-3192637f2aad None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:15:58.894240 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9166de2b-9c18-4d85-96da-2b77aa639a05 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '952911c492444ffea4b211a24b564117', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:15:58.894898 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9166de2b-9c18-4d85-96da-2b77aa639a05 service neutron] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:15:58.895179 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9166de2b-9c18-4d85-96da-2b77aa639a05 service neutron] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:15:58.895478 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9166de2b-9c18-4d85-96da-2b77aa639a05 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:15:58.904933 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9166de2b-9c18-4d85-96da-2b77aa639a05 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:15:58.906502 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9166de2b-9c18-4d85-96da-2b77aa639a05 service neutron] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:15:58.916518 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 771/1530] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:15:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:16:00.934062 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00347828-9bf3-43f3-a5a1-9324ace1f190 None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:16:00.935024 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00347828-9bf3-43f3-a5a1-9324ace1f190 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:16:00.935024 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00347828-9bf3-43f3-a5a1-9324ace1f190 None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:16:00.937250 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 760/1531] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 14 09:16:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:16:00.943692 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067d3eae-cfb5-4060-8197-be74406b9092 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:16:00.944471 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067d3eae-cfb5-4060-8197-be74406b9092 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:16:00.944658 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067d3eae-cfb5-4060-8197-be74406b9092 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:16:00.970983 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-067d3eae-cfb5-4060-8197-be74406b9092 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:16:00.972217 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-067d3eae-cfb5-4060-8197-be74406b9092 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:16:00.981659 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 772/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 14 09:16:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:16:01.017313 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00347828-9bf3-43f3-a5a1-9324ace1f190 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:16:01.030968 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00347828-9bf3-43f3-a5a1-9324ace1f190 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:16:01.034959 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4eddb7b-abc2-45c0-89d3-1a449e333aee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '952911c492444ffea4b211a24b564117', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:16:01.035637 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4eddb7b-abc2-45c0-89d3-1a449e333aee service neutron] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:16:01.035920 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4eddb7b-abc2-45c0-89d3-1a449e333aee service neutron] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:16:01.036197 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4eddb7b-abc2-45c0-89d3-1a449e333aee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:16:01.046105 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4eddb7b-abc2-45c0-89d3-1a449e333aee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:16:01.047608 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4eddb7b-abc2-45c0-89d3-1a449e333aee service neutron] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:16:01.057324 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 761/1533] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:16:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:16:04.553828 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6e56fc-0c89-4adb-ad1e-41473adb0071 None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:16:04.554269 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6e56fc-0c89-4adb-ad1e-41473adb0071 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:16:04.554450 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6e56fc-0c89-4adb-ad1e-41473adb0071 None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:16:04.556729 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 773/1534] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 14 09:16:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:16:04.562390 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e7a1da-b62d-4082-94a6-a0415e914ac1 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:16:04.562869 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e7a1da-b62d-4082-94a6-a0415e914ac1 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:16:04.562997 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e7a1da-b62d-4082-94a6-a0415e914ac1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:16:04.583169 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-01e7a1da-b62d-4082-94a6-a0415e914ac1 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:16:04.584402 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-01e7a1da-b62d-4082-94a6-a0415e914ac1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:16:04.593343 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 762/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 14 09:16:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 09:16:04.894152 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6e56fc-0c89-4adb-ad1e-41473adb0071 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:16:04.896466 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-3d6e56fc-0c89-4adb-ad1e-41473adb0071 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:16:04.967065 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:16:04.983535 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6e56fc-0c89-4adb-ad1e-41473adb0071 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:16:04.987403 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e4dfdb-412f-462d-97b0-045d9c295eff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e76f1952a5b146808844b9f2b7395035', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:16:04.988059 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e4dfdb-412f-462d-97b0-045d9c295eff service cinder] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:16:04.988341 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e4dfdb-412f-462d-97b0-045d9c295eff service cinder] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:16:04.988617 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e4dfdb-412f-462d-97b0-045d9c295eff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:16:04.997892 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e4dfdb-412f-462d-97b0-045d9c295eff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:16:04.999374 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e4dfdb-412f-462d-97b0-045d9c295eff service cinder] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:16:05.009516 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 774/1536] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:16:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:16:07.040227 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf5afcb-232b-4c0c-a810-27f54d675cb3 None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:16:07.041103 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf5afcb-232b-4c0c-a810-27f54d675cb3 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:16:07.041103 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf5afcb-232b-4c0c-a810-27f54d675cb3 None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:16:07.043257 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 763/1537] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 14 09:16:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:16:07.052313 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f76464-334c-45f7-8b63-ecb305545946 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:16:07.052758 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f76464-334c-45f7-8b63-ecb305545946 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:16:07.052875 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f76464-334c-45f7-8b63-ecb305545946 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:16:07.073529 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-07f76464-334c-45f7-8b63-ecb305545946 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:16:07.074715 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-07f76464-334c-45f7-8b63-ecb305545946 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:16:07.086353 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 775/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 14 09:16:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:16:07.388157 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf5afcb-232b-4c0c-a810-27f54d675cb3 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:16:07.390280 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-1bf5afcb-232b-4c0c-a810-27f54d675cb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:16:07.408775 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf5afcb-232b-4c0c-a810-27f54d675cb3 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:16:07.415982 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf50e63f-fa6c-4b89-b09d-c81a43b8cce9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e76f1952a5b146808844b9f2b7395035', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:16:07.416640 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf50e63f-fa6c-4b89-b09d-c81a43b8cce9 service cinder] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:16:07.416930 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf50e63f-fa6c-4b89-b09d-c81a43b8cce9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:16:07.417206 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf50e63f-fa6c-4b89-b09d-c81a43b8cce9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:16:07.427154 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf50e63f-fa6c-4b89-b09d-c81a43b8cce9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:16:07.428648 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf50e63f-fa6c-4b89-b09d-c81a43b8cce9 service cinder] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:16:07.438509 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 764/1539] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:16:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:16:17.068509 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=11 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:16:17.434390 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=89 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:17:34.883872 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30ddbc4-76b1-4241-b7e3-6243d58c96cd None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:17:34.885540 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30ddbc4-76b1-4241-b7e3-6243d58c96cd None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:17:34.885540 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30ddbc4-76b1-4241-b7e3-6243d58c96cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:17:34.914902 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-b30ddbc4-76b1-4241-b7e3-6243d58c96cd None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:17:34.916158 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-b30ddbc4-76b1-4241-b7e3-6243d58c96cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:17:34.928149 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 776/1540] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:17:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:17:34.937498 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dc9745-faf1-430e-9c80-f156d7d89ddc None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:17:34.938514 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dc9745-faf1-430e-9c80-f156d7d89ddc None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:17:34.939012 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dc9745-faf1-430e-9c80-f156d7d89ddc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:17:34.966738 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-02dc9745-faf1-430e-9c80-f156d7d89ddc None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:17:34.968004 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-02dc9745-faf1-430e-9c80-f156d7d89ddc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:17:34.980230 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 765/1541] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:17:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:17:34.985422 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30ddbc4-76b1-4241-b7e3-6243d58c96cd None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:17:34.985763 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc36672f-9027-432f-bf29-f59ddf9658a9 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:17:34.986050 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc36672f-9027-432f-bf29-f59ddf9658a9 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:17:34.986326 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc36672f-9027-432f-bf29-f59ddf9658a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:17:34.996166 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30ddbc4-76b1-4241-b7e3-6243d58c96cd None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:17:35.012529 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d724b2d-921c-4b9a-860e-b629cf3b439f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:17:35.013317 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d724b2d-921c-4b9a-860e-b629cf3b439f admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:17:35.014038 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d724b2d-921c-4b9a-860e-b629cf3b439f admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:17:35.014426 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d724b2d-921c-4b9a-860e-b629cf3b439f admin admin] PATH_INFO: `/v3/domains` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:17:35.015495 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d724b2d-921c-4b9a-860e-b629cf3b439f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:17:35.015865 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d724b2d-921c-4b9a-860e-b629cf3b439f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:17:35.018519 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d724b2d-921c-4b9a-860e-b629cf3b439f admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:17:35.030868 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-bc36672f-9027-432f-bf29-f59ddf9658a9 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:17:35.032113 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-bc36672f-9027-432f-bf29-f59ddf9658a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:17:35.042203 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 777/1542] 10.222.0.26 () {58 vars in 1152 bytes} [Thu Jul 14 09:17:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:17:35.050815 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d724b2d-921c-4b9a-860e-b629cf3b439f admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:17:35.052553 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 766/1543] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:17:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 09:17:35.061442 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c439ba0-b735-499c-b9b3-ded56f78d29c None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:17:35.061742 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c439ba0-b735-499c-b9b3-ded56f78d29c None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:17:35.061995 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c439ba0-b735-499c-b9b3-ded56f78d29c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:17:35.069915 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d724b2d-921c-4b9a-860e-b629cf3b439f admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:17:35.088709 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77f6a48-e9ee-4870-9874-b28127607f01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:17:35.089659 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77f6a48-e9ee-4870-9874-b28127607f01 admin admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:17:35.089942 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77f6a48-e9ee-4870-9874-b28127607f01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:17:35.090215 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77f6a48-e9ee-4870-9874-b28127607f01 admin admin] PATH_INFO: `/v3/projects` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:17:35.090965 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77f6a48-e9ee-4870-9874-b28127607f01 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:17:35.094471 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: WARNING py.warnings [None req-a77f6a48-e9ee-4870-9874-b28127607f01 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 14 09:17:35.094471 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: warnings.warn(msg) Jul 14 09:17:35.094471 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: Jul 14 09:17:35.102102 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77f6a48-e9ee-4870-9874-b28127607f01 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:17:35.106031 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-3c439ba0-b735-499c-b9b3-ded56f78d29c None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:17:35.107802 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-3c439ba0-b735-499c-b9b3-ded56f78d29c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:17:35.131102 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 767/1544] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:17:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 09:17:35.132452 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639127be-78c6-423b-b278-5daaab44c429 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:17:35.132739 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639127be-78c6-423b-b278-5daaab44c429 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:17:35.133019 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639127be-78c6-423b-b278-5daaab44c429 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:17:35.167546 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-639127be-78c6-423b-b278-5daaab44c429 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:17:35.168750 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-639127be-78c6-423b-b278-5daaab44c429 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:17:35.173765 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 778/1545] 10.222.0.26 () {60 vars in 1152 bytes} [Thu Jul 14 09:17:35 2022] POST /identity/v3/projects => generated 375 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 09:17:35.175518 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650a67e6-d5ce-466b-b3c4-d8ffa039f361 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:17:35.176079 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650a67e6-d5ce-466b-b3c4-d8ffa039f361 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:17:35.176588 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650a67e6-d5ce-466b-b3c4-d8ffa039f361 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:17:35.181654 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 768/1546] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:17:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:17:35.182661 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639127be-78c6-423b-b278-5daaab44c429 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:17:35.199961 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639127be-78c6-423b-b278-5daaab44c429 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:17:35.205172 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-650a67e6-d5ce-466b-b3c4-d8ffa039f361 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:17:35.207054 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-650a67e6-d5ce-466b-b3c4-d8ffa039f361 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:17:35.303730 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbab530-7d70-4174-854d-8ccf39168b7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:17:35.304433 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbab530-7d70-4174-854d-8ccf39168b7c admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:17:35.304722 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbab530-7d70-4174-854d-8ccf39168b7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:17:35.305137 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbab530-7d70-4174-854d-8ccf39168b7c admin admin] PATH_INFO: `/v3/domains` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:17:35.305889 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbab530-7d70-4174-854d-8ccf39168b7c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:17:35.306201 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbab530-7d70-4174-854d-8ccf39168b7c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:17:35.307317 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: WARNING py.warnings [None req-2bbab530-7d70-4174-854d-8ccf39168b7c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 14 09:17:35.307317 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: warnings.warn(msg) Jul 14 09:17:35.307317 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: Jul 14 09:17:35.308213 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 779/1547] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:17:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:17:35.309625 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fab57e-2905-4420-9dbf-51ea582cf6f3 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:17:35.309916 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fab57e-2905-4420-9dbf-51ea582cf6f3 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:17:35.310193 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fab57e-2905-4420-9dbf-51ea582cf6f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:17:35.311622 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbab530-7d70-4174-854d-8ccf39168b7c admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:17:35.327978 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 769/1548] 10.222.0.26 () {58 vars in 1152 bytes} [Thu Jul 14 09:17:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:17:35.328958 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-68fab57e-2905-4420-9dbf-51ea582cf6f3 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:17:35.329608 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbab530-7d70-4174-854d-8ccf39168b7c admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:17:35.330198 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-68fab57e-2905-4420-9dbf-51ea582cf6f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:17:35.338687 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 780/1549] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:17:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:17:35.340136 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e914b8-2ea5-44ac-aa36-514428d1cb1d None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:17:35.340424 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e914b8-2ea5-44ac-aa36-514428d1cb1d None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:17:35.340687 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbab530-7d70-4174-854d-8ccf39168b7c admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:17:35.340807 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e914b8-2ea5-44ac-aa36-514428d1cb1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:17:35.345338 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae40ff1e-3cd7-42d3-b52e-79c00a1dc257 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:17:35.346406 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae40ff1e-3cd7-42d3-b52e-79c00a1dc257 admin admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:17:35.346702 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae40ff1e-3cd7-42d3-b52e-79c00a1dc257 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:17:35.346978 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae40ff1e-3cd7-42d3-b52e-79c00a1dc257 admin admin] PATH_INFO: `/v3/users` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:17:35.347711 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae40ff1e-3cd7-42d3-b52e-79c00a1dc257 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:17:35.348759 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: WARNING py.warnings [None req-ae40ff1e-3cd7-42d3-b52e-79c00a1dc257 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 14 09:17:35.348759 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: warnings.warn(msg) Jul 14 09:17:35.348759 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: Jul 14 09:17:35.349970 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae40ff1e-3cd7-42d3-b52e-79c00a1dc257 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:17:35.358331 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-c9e914b8-2ea5-44ac-aa36-514428d1cb1d None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:17:35.359554 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-c9e914b8-2ea5-44ac-aa36-514428d1cb1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:17:35.368426 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 781/1550] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:17:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:17:35.369973 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db7b980-c545-416c-bb00-2a8fb6b6035d None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:17:35.370410 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db7b980-c545-416c-bb00-2a8fb6b6035d None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:17:35.370862 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db7b980-c545-416c-bb00-2a8fb6b6035d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:17:35.371365 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 770/1551] 10.222.0.26 () {60 vars in 1143 bytes} [Thu Jul 14 09:17:35 2022] POST /identity/v3/users => generated 288 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 09:17:35.372725 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ad80ff-f39e-4212-912f-64981ceb8234 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:17:35.373017 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ad80ff-f39e-4212-912f-64981ceb8234 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:17:35.373291 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ad80ff-f39e-4212-912f-64981ceb8234 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:17:35.391050 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-47ad80ff-f39e-4212-912f-64981ceb8234 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:17:35.391595 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-1db7b980-c545-416c-bb00-2a8fb6b6035d None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:17:35.392624 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-47ad80ff-f39e-4212-912f-64981ceb8234 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:17:35.392733 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-1db7b980-c545-416c-bb00-2a8fb6b6035d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:17:35.407814 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 771/1553] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:17:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:17:35.408028 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 782/1553] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:17:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:17:35.408138 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c98405-7e9a-475c-acae-606516f7de6d None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:17:35.408138 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c98405-7e9a-475c-acae-606516f7de6d None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:17:35.408138 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c98405-7e9a-475c-acae-606516f7de6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:17:35.408848 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219c34f6-b59d-4307-b53a-6b331c33b82f None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:17:35.409139 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219c34f6-b59d-4307-b53a-6b331c33b82f None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:17:35.409417 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219c34f6-b59d-4307-b53a-6b331c33b82f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:17:35.420910 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-42c98405-7e9a-475c-acae-606516f7de6d None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:17:35.422031 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-42c98405-7e9a-475c-acae-606516f7de6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:17:35.427380 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-219c34f6-b59d-4307-b53a-6b331c33b82f None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:17:35.428532 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-219c34f6-b59d-4307-b53a-6b331c33b82f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:17:35.430119 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 772/1554] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:17:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:17:35.430862 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c98405-7e9a-475c-acae-606516f7de6d None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:17:35.439023 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 783/1555] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:17:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:17:35.440740 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c98405-7e9a-475c-acae-606516f7de6d None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:17:35.440966 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccb6768-4eda-4ea1-ada1-3bb4a1a35535 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:17:35.441257 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccb6768-4eda-4ea1-ada1-3bb4a1a35535 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:17:35.441533 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccb6768-4eda-4ea1-ada1-3bb4a1a35535 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:17:35.445416 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4f7fee-102d-4981-adf8-9835f3a15669 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:17:35.446074 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4f7fee-102d-4981-adf8-9835f3a15669 admin admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:17:35.446362 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4f7fee-102d-4981-adf8-9835f3a15669 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:17:35.446701 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4f7fee-102d-4981-adf8-9835f3a15669 admin admin] PATH_INFO: `/v3/projects` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:17:35.447410 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4f7fee-102d-4981-adf8-9835f3a15669 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:17:35.449359 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4f7fee-102d-4981-adf8-9835f3a15669 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:17:35.460376 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-2ccb6768-4eda-4ea1-ada1-3bb4a1a35535 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:17:35.461562 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-2ccb6768-4eda-4ea1-ada1-3bb4a1a35535 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:17:35.473793 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 784/1556] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:17:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:17:35.473793 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccb6768-4eda-4ea1-ada1-3bb4a1a35535 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:17:35.481007 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 773/1557] 10.222.0.26 () {60 vars in 1152 bytes} [Thu Jul 14 09:17:35 2022] POST /identity/v3/projects => generated 377 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 09:17:35.481007 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4f7fee-102d-4981-adf8-9835f3a15669 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:17:35.481263 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccb6768-4eda-4ea1-ada1-3bb4a1a35535 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:17:35.536774 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c90394-7b78-419b-b993-736ff3294939 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:17:35.537443 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c90394-7b78-419b-b993-736ff3294939 admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:17:35.537725 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c90394-7b78-419b-b993-736ff3294939 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:17:35.538001 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c90394-7b78-419b-b993-736ff3294939 admin admin] PATH_INFO: `/v3/domains` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:17:35.538914 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c90394-7b78-419b-b993-736ff3294939 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:17:35.539218 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c90394-7b78-419b-b993-736ff3294939 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:17:35.540896 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c90394-7b78-419b-b993-736ff3294939 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:17:35.545868 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4f7fee-102d-4981-adf8-9835f3a15669 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:17:35.553054 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 785/1558] 10.222.0.26 () {58 vars in 1152 bytes} [Thu Jul 14 09:17:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:17:35.554331 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442b2a69-ff97-43d0-94aa-423819128f65 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:17:35.554618 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442b2a69-ff97-43d0-94aa-423819128f65 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:17:35.554899 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442b2a69-ff97-43d0-94aa-423819128f65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:17:35.560634 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1b789c-444d-44ab-8548-fd0408865c61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:17:35.561439 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1b789c-444d-44ab-8548-fd0408865c61 admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:17:35.561805 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1b789c-444d-44ab-8548-fd0408865c61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:17:35.562122 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1b789c-444d-44ab-8548-fd0408865c61 admin admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:17:35.562926 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1b789c-444d-44ab-8548-fd0408865c61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:17:35.564766 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1b789c-444d-44ab-8548-fd0408865c61 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:17:35.575479 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 774/1559] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jul 14 09:17:35 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 09:17:35.575912 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-442b2a69-ff97-43d0-94aa-423819128f65 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:17:35.576735 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535cf495-c251-4c95-8293-496bbed46083 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:17:35.577022 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535cf495-c251-4c95-8293-496bbed46083 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:17:35.577297 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535cf495-c251-4c95-8293-496bbed46083 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:17:35.577467 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-442b2a69-ff97-43d0-94aa-423819128f65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:17:35.588994 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 786/1560] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:17:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:17:35.588994 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442b2a69-ff97-43d0-94aa-423819128f65 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:17:35.594830 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:17:35.595404 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-535cf495-c251-4c95-8293-496bbed46083 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:17:35.596736 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-535cf495-c251-4c95-8293-496bbed46083 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:17:35.598953 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442b2a69-ff97-43d0-94aa-423819128f65 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:17:35.604698 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2220cef8-1944-471a-b0ab-947932815ff7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:17:35.605271 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 775/1561] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:17:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:17:35.605403 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2220cef8-1944-471a-b0ab-947932815ff7 admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:17:35.605631 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2220cef8-1944-471a-b0ab-947932815ff7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:17:35.605930 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2220cef8-1944-471a-b0ab-947932815ff7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:17:35.606475 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73de333d-2262-42c1-8d63-fd2aecbf8aa7 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:17:35.606682 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2220cef8-1944-471a-b0ab-947932815ff7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:17:35.606796 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73de333d-2262-42c1-8d63-fd2aecbf8aa7 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:17:35.606958 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2220cef8-1944-471a-b0ab-947932815ff7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:17:35.607064 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73de333d-2262-42c1-8d63-fd2aecbf8aa7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:17:35.608866 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2220cef8-1944-471a-b0ab-947932815ff7 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:17:35.614125 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:17:35.622225 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 787/1562] 10.222.0.26 () {58 vars in 1152 bytes} [Thu Jul 14 09:17:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:17:35.622970 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2220cef8-1944-471a-b0ab-947932815ff7 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:17:35.627798 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-73de333d-2262-42c1-8d63-fd2aecbf8aa7 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:17:35.628961 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-73de333d-2262-42c1-8d63-fd2aecbf8aa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:17:35.633350 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2220cef8-1944-471a-b0ab-947932815ff7 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:17:35.637792 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 776/1563] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:17:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:17:35.637933 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11b2069-4f7b-4acb-8b05-a29870d7aa72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:17:35.638545 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11b2069-4f7b-4acb-8b05-a29870d7aa72 admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:17:35.638662 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73de333d-2262-42c1-8d63-fd2aecbf8aa7 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:17:35.638851 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11b2069-4f7b-4acb-8b05-a29870d7aa72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:17:35.639102 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11b2069-4f7b-4acb-8b05-a29870d7aa72 admin admin] PATH_INFO: `/v3/domains` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:17:35.639893 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11b2069-4f7b-4acb-8b05-a29870d7aa72 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:17:35.640200 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11b2069-4f7b-4acb-8b05-a29870d7aa72 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:17:35.642124 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11b2069-4f7b-4acb-8b05-a29870d7aa72 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:17:35.650563 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73de333d-2262-42c1-8d63-fd2aecbf8aa7 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:17:35.661766 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 788/1564] 10.222.0.26 () {58 vars in 1152 bytes} [Thu Jul 14 09:17:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:17:35.661766 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11b2069-4f7b-4acb-8b05-a29870d7aa72 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:17:35.662055 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca3ff9d-c9d9-4f1d-a8ce-67fc52231b28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:17:35.662055 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca3ff9d-c9d9-4f1d-a8ce-67fc52231b28 admin admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:17:35.662055 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca3ff9d-c9d9-4f1d-a8ce-67fc52231b28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:17:35.662055 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca3ff9d-c9d9-4f1d-a8ce-67fc52231b28 admin admin] PATH_INFO: `/v3/users` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:17:35.662470 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca3ff9d-c9d9-4f1d-a8ce-67fc52231b28 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:17:35.662470 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca3ff9d-c9d9-4f1d-a8ce-67fc52231b28 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:17:35.664953 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11b2069-4f7b-4acb-8b05-a29870d7aa72 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:17:35.670000 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c07542c-b150-49a2-98b2-b20b0f129f39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:17:35.670700 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c07542c-b150-49a2-98b2-b20b0f129f39 admin admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:17:35.671039 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c07542c-b150-49a2-98b2-b20b0f129f39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:17:35.671279 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c07542c-b150-49a2-98b2-b20b0f129f39 admin admin] PATH_INFO: `/v3/projects` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:17:35.672035 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c07542c-b150-49a2-98b2-b20b0f129f39 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:17:35.673996 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c07542c-b150-49a2-98b2-b20b0f129f39 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:17:35.680626 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 777/1565] 10.222.0.26 () {60 vars in 1143 bytes} [Thu Jul 14 09:17:35 2022] POST /identity/v3/users => generated 289 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 09:17:35.684420 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca3ff9d-c9d9-4f1d-a8ce-67fc52231b28 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:17:35.694070 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca3ff9d-c9d9-4f1d-a8ce-67fc52231b28 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:17:35.698625 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5594c3c2-b435-4971-aab9-0db8ddedf9ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:17:35.699343 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5594c3c2-b435-4971-aab9-0db8ddedf9ba admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:17:35.699602 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5594c3c2-b435-4971-aab9-0db8ddedf9ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:17:35.699880 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5594c3c2-b435-4971-aab9-0db8ddedf9ba admin admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:17:35.700608 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5594c3c2-b435-4971-aab9-0db8ddedf9ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:17:35.702251 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5594c3c2-b435-4971-aab9-0db8ddedf9ba admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:17:35.702436 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 789/1566] 10.222.0.26 () {60 vars in 1152 bytes} [Thu Jul 14 09:17:35 2022] POST /identity/v3/projects => generated 391 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 09:17:35.703678 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c07542c-b150-49a2-98b2-b20b0f129f39 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:17:35.715720 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 778/1567] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jul 14 09:17:35 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 09:17:35.715720 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5594c3c2-b435-4971-aab9-0db8ddedf9ba admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:17:35.722775 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5594c3c2-b435-4971-aab9-0db8ddedf9ba admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:17:35.767386 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c07542c-b150-49a2-98b2-b20b0f129f39 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:17:35.772167 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322547ae-43fd-456e-9cef-445d07baefd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:17:35.772830 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322547ae-43fd-456e-9cef-445d07baefd7 admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:17:35.773110 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322547ae-43fd-456e-9cef-445d07baefd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:17:35.773344 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322547ae-43fd-456e-9cef-445d07baefd7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:17:35.774069 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322547ae-43fd-456e-9cef-445d07baefd7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:17:35.774401 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322547ae-43fd-456e-9cef-445d07baefd7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:17:35.776460 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322547ae-43fd-456e-9cef-445d07baefd7 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:17:35.779392 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9b93ba-ba1e-4286-878b-1175aa28e760 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:17:35.780075 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9b93ba-ba1e-4286-878b-1175aa28e760 admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:17:35.780361 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9b93ba-ba1e-4286-878b-1175aa28e760 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:17:35.780631 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9b93ba-ba1e-4286-878b-1175aa28e760 admin admin] PATH_INFO: `/v3/domains` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:17:35.781379 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9b93ba-ba1e-4286-878b-1175aa28e760 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:17:35.781677 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9b93ba-ba1e-4286-878b-1175aa28e760 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:17:35.783291 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9b93ba-ba1e-4286-878b-1175aa28e760 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:17:35.788515 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 790/1568] 10.222.0.26 () {58 vars in 1152 bytes} [Thu Jul 14 09:17:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:17:35.795478 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 779/1569] 10.222.0.26 () {58 vars in 1152 bytes} [Thu Jul 14 09:17:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:17:35.796174 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322547ae-43fd-456e-9cef-445d07baefd7 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:17:35.796579 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9b93ba-ba1e-4286-878b-1175aa28e760 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:17:35.806487 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9b93ba-ba1e-4286-878b-1175aa28e760 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:17:35.807092 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322547ae-43fd-456e-9cef-445d07baefd7 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:17:35.811216 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d729d67-6595-4079-80eb-06093572f332 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:17:35.811869 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d729d67-6595-4079-80eb-06093572f332 admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:17:35.811993 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f492b8-9e70-4186-bf7f-ee9e7162042e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:17:35.812205 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d729d67-6595-4079-80eb-06093572f332 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:17:35.812445 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d729d67-6595-4079-80eb-06093572f332 admin admin] PATH_INFO: `/v3/domains` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:17:35.812623 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f492b8-9e70-4186-bf7f-ee9e7162042e admin admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:17:35.812918 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f492b8-9e70-4186-bf7f-ee9e7162042e admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:17:35.813158 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d729d67-6595-4079-80eb-06093572f332 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:17:35.813354 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f492b8-9e70-4186-bf7f-ee9e7162042e admin admin] PATH_INFO: `/v3/projects` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:17:35.813449 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d729d67-6595-4079-80eb-06093572f332 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:17:35.813955 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f492b8-9e70-4186-bf7f-ee9e7162042e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:17:35.815029 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d729d67-6595-4079-80eb-06093572f332 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:17:35.815946 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f492b8-9e70-4186-bf7f-ee9e7162042e admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:17:35.827224 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 780/1570] 10.222.0.26 () {58 vars in 1152 bytes} [Thu Jul 14 09:17:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:17:35.828108 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d729d67-6595-4079-80eb-06093572f332 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:17:35.837848 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d729d67-6595-4079-80eb-06093572f332 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:17:35.842356 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98437cfb-b175-4ba6-929c-6bbfc6a5c96f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:17:35.842721 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 791/1571] 10.222.0.26 () {60 vars in 1152 bytes} [Thu Jul 14 09:17:35 2022] POST /identity/v3/projects => generated 379 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 09:17:35.842993 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98437cfb-b175-4ba6-929c-6bbfc6a5c96f admin admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:17:35.843318 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98437cfb-b175-4ba6-929c-6bbfc6a5c96f admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:17:35.843583 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98437cfb-b175-4ba6-929c-6bbfc6a5c96f admin admin] PATH_INFO: `/v3/projects` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:17:35.843698 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f492b8-9e70-4186-bf7f-ee9e7162042e admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:17:35.844301 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98437cfb-b175-4ba6-929c-6bbfc6a5c96f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:17:35.846230 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98437cfb-b175-4ba6-929c-6bbfc6a5c96f admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:17:35.875821 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 781/1572] 10.222.0.26 () {60 vars in 1152 bytes} [Thu Jul 14 09:17:35 2022] POST /identity/v3/projects => generated 377 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 09:17:35.876897 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98437cfb-b175-4ba6-929c-6bbfc6a5c96f admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:17:35.905493 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f492b8-9e70-4186-bf7f-ee9e7162042e admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:17:35.942582 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:17:35.946682 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98437cfb-b175-4ba6-929c-6bbfc6a5c96f admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:17:35.951328 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4389ed01-c8a3-4185-92a0-fe404ae20613 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:17:35.952041 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4389ed01-c8a3-4185-92a0-fe404ae20613 admin admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:17:35.952294 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4389ed01-c8a3-4185-92a0-fe404ae20613 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:17:35.952569 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4389ed01-c8a3-4185-92a0-fe404ae20613 admin admin] PATH_INFO: `/v3/users` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:17:35.953252 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4389ed01-c8a3-4185-92a0-fe404ae20613 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:17:35.955331 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4389ed01-c8a3-4185-92a0-fe404ae20613 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:17:35.971635 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3544551f-d47e-434e-94fa-e2f7eb79217e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:17:35.972312 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3544551f-d47e-434e-94fa-e2f7eb79217e admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:17:35.972595 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3544551f-d47e-434e-94fa-e2f7eb79217e admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:17:35.972871 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3544551f-d47e-434e-94fa-e2f7eb79217e admin admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:17:35.973599 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3544551f-d47e-434e-94fa-e2f7eb79217e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:17:35.975125 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 782/1573] 10.222.0.26 () {60 vars in 1143 bytes} [Thu Jul 14 09:17:35 2022] POST /identity/v3/users => generated 296 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 09:17:35.975287 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3544551f-d47e-434e-94fa-e2f7eb79217e admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:17:35.975934 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4389ed01-c8a3-4185-92a0-fe404ae20613 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:17:35.985455 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4389ed01-c8a3-4185-92a0-fe404ae20613 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:17:35.990580 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 792/1574] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jul 14 09:17:35 2022] GET /identity/v3/roles => generated 1420 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 09:17:35.990580 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3544551f-d47e-434e-94fa-e2f7eb79217e admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:17:35.990845 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e1d429-dcb0-4470-bb38-bc5c46500850 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:17:35.990845 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e1d429-dcb0-4470-bb38-bc5c46500850 admin admin] REQUEST_METHOD: `PUT` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:17:35.991117 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e1d429-dcb0-4470-bb38-bc5c46500850 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:17:35.991425 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e1d429-dcb0-4470-bb38-bc5c46500850 admin admin] PATH_INFO: `/v3/projects/728ef81aa26445e6a42abdf976d146db/users/4d619bfc0939493480fa98451db2f30f/roles/d1691322ce6a4edab7ce722edf0f2988` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:17:35.998516 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3544551f-d47e-434e-94fa-e2f7eb79217e admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:17:36.003748 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b879e08e-a4ce-4076-941e-64ce3757b454 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:17:36.004387 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b879e08e-a4ce-4076-941e-64ce3757b454 admin admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:17:36.004671 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b879e08e-a4ce-4076-941e-64ce3757b454 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:17:36.004942 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b879e08e-a4ce-4076-941e-64ce3757b454 admin admin] PATH_INFO: `/v3/projects` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:17:36.005680 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b879e08e-a4ce-4076-941e-64ce3757b454 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:17:36.006878 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e1d429-dcb0-4470-bb38-bc5c46500850 admin admin] RBAC: Authorizing `identity:create_grant(project_id=728ef81aa26445e6a42abdf976d146db, user_id=4d619bfc0939493480fa98451db2f30f, role_id=d1691322ce6a4edab7ce722edf0f2988)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:17:36.007800 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b879e08e-a4ce-4076-941e-64ce3757b454 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:17:36.009975 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e1d429-dcb0-4470-bb38-bc5c46500850 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:17:36.012279 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:17:36.025902 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 783/1575] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jul 14 09:17:35 2022] PUT /identity/v3/projects/728ef81aa26445e6a42abdf976d146db/users/4d619bfc0939493480fa98451db2f30f/roles/d1691322ce6a4edab7ce722edf0f2988 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:17:36.026794 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e1d429-dcb0-4470-bb38-bc5c46500850 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:17:36.035446 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 793/1576] 10.222.0.26 () {60 vars in 1152 bytes} [Thu Jul 14 09:17:35 2022] POST /identity/v3/projects => generated 385 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 09:17:36.036443 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b879e08e-a4ce-4076-941e-64ce3757b454 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:17:36.091975 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e1d429-dcb0-4470-bb38-bc5c46500850 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:17:36.096568 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960ab5c4-d165-443c-a2e8-18afa79e5d2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:17:36.097230 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960ab5c4-d165-443c-a2e8-18afa79e5d2f admin admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:17:36.097344 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b879e08e-a4ce-4076-941e-64ce3757b454 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:17:36.097514 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960ab5c4-d165-443c-a2e8-18afa79e5d2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:17:36.097788 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960ab5c4-d165-443c-a2e8-18afa79e5d2f admin admin] PATH_INFO: `/v3/projects` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:17:36.098479 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960ab5c4-d165-443c-a2e8-18afa79e5d2f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:17:36.101018 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960ab5c4-d165-443c-a2e8-18afa79e5d2f admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:17:36.101831 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052ac3fd-7a10-4a73-b6a5-520d5cd7fc04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:17:36.102503 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052ac3fd-7a10-4a73-b6a5-520d5cd7fc04 admin admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:17:36.102804 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052ac3fd-7a10-4a73-b6a5-520d5cd7fc04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:17:36.103065 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052ac3fd-7a10-4a73-b6a5-520d5cd7fc04 admin admin] PATH_INFO: `/v3/projects` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:17:36.103793 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052ac3fd-7a10-4a73-b6a5-520d5cd7fc04 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:17:36.105725 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052ac3fd-7a10-4a73-b6a5-520d5cd7fc04 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:17:36.128064 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 784/1577] 10.222.0.26 () {60 vars in 1152 bytes} [Thu Jul 14 09:17:36 2022] POST /identity/v3/projects => generated 373 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 09:17:36.128728 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960ab5c4-d165-443c-a2e8-18afa79e5d2f admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:17:36.135773 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 794/1578] 10.222.0.26 () {60 vars in 1152 bytes} [Thu Jul 14 09:17:36 2022] POST /identity/v3/projects => generated 371 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 09:17:36.136599 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052ac3fd-7a10-4a73-b6a5-520d5cd7fc04 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:17:36.192909 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960ab5c4-d165-443c-a2e8-18afa79e5d2f admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:17:36.197425 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7951c5b7-1fad-4741-8a01-be34b90c8dd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:17:36.198123 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7951c5b7-1fad-4741-8a01-be34b90c8dd2 admin admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:17:36.198409 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7951c5b7-1fad-4741-8a01-be34b90c8dd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:17:36.198685 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7951c5b7-1fad-4741-8a01-be34b90c8dd2 admin admin] PATH_INFO: `/v3/users` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:17:36.199445 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7951c5b7-1fad-4741-8a01-be34b90c8dd2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:17:36.199738 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052ac3fd-7a10-4a73-b6a5-520d5cd7fc04 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:17:36.201355 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7951c5b7-1fad-4741-8a01-be34b90c8dd2 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:17:36.204696 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9193c282-9dbf-444d-8321-8df27cfa6ceb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:17:36.205368 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9193c282-9dbf-444d-8321-8df27cfa6ceb admin admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:17:36.205648 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9193c282-9dbf-444d-8321-8df27cfa6ceb admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:17:36.205921 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9193c282-9dbf-444d-8321-8df27cfa6ceb admin admin] PATH_INFO: `/v3/users` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:17:36.206605 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9193c282-9dbf-444d-8321-8df27cfa6ceb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:17:36.208538 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9193c282-9dbf-444d-8321-8df27cfa6ceb admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:17:36.225708 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 785/1579] 10.222.0.26 () {60 vars in 1143 bytes} [Thu Jul 14 09:17:36 2022] POST /identity/v3/users => generated 290 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 09:17:36.226529 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7951c5b7-1fad-4741-8a01-be34b90c8dd2 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:17:36.232878 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 795/1580] 10.222.0.26 () {60 vars in 1143 bytes} [Thu Jul 14 09:17:36 2022] POST /identity/v3/users => generated 289 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 09:17:36.234237 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9193c282-9dbf-444d-8321-8df27cfa6ceb admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:17:36.239492 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7951c5b7-1fad-4741-8a01-be34b90c8dd2 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:17:36.244043 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9193c282-9dbf-444d-8321-8df27cfa6ceb admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:17:36.244334 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218835a2-894b-4dc5-9eb9-168bcbf4fd6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:17:36.245063 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218835a2-894b-4dc5-9eb9-168bcbf4fd6e admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:17:36.245351 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218835a2-894b-4dc5-9eb9-168bcbf4fd6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:17:36.245633 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218835a2-894b-4dc5-9eb9-168bcbf4fd6e admin admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:17:36.246420 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218835a2-894b-4dc5-9eb9-168bcbf4fd6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:17:36.248061 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218835a2-894b-4dc5-9eb9-168bcbf4fd6e admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:17:36.249154 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31ef536-f80b-41c6-8d79-4f1c86e9fd77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:17:36.249816 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31ef536-f80b-41c6-8d79-4f1c86e9fd77 admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:17:36.250125 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31ef536-f80b-41c6-8d79-4f1c86e9fd77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:17:36.250401 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31ef536-f80b-41c6-8d79-4f1c86e9fd77 admin admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:17:36.251125 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31ef536-f80b-41c6-8d79-4f1c86e9fd77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:17:36.252746 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31ef536-f80b-41c6-8d79-4f1c86e9fd77 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:17:36.259143 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 786/1581] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jul 14 09:17:36 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 09:17:36.260443 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f99cd58-7513-4168-8800-99ee123d7b53 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:17:36.260731 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f99cd58-7513-4168-8800-99ee123d7b53 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:17:36.261010 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f99cd58-7513-4168-8800-99ee123d7b53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:17:36.266724 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 796/1582] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jul 14 09:17:36 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 09:17:36.266724 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31ef536-f80b-41c6-8d79-4f1c86e9fd77 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:17:36.274640 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31ef536-f80b-41c6-8d79-4f1c86e9fd77 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:17:36.279833 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e625f2-1837-4e46-a87d-12dea996a2e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:17:36.280000 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-3f99cd58-7513-4168-8800-99ee123d7b53 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:17:36.280541 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e625f2-1837-4e46-a87d-12dea996a2e7 admin admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:17:36.280819 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e625f2-1837-4e46-a87d-12dea996a2e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:17:36.281060 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-3f99cd58-7513-4168-8800-99ee123d7b53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:17:36.281182 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e625f2-1837-4e46-a87d-12dea996a2e7 admin admin] PATH_INFO: `/v3/users` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:17:36.281759 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e625f2-1837-4e46-a87d-12dea996a2e7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:17:36.283721 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e625f2-1837-4e46-a87d-12dea996a2e7 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:17:36.288836 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 787/1583] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:17:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:17:36.289582 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f99cd58-7513-4168-8800-99ee123d7b53 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:17:36.299221 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:17:36.300434 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f99cd58-7513-4168-8800-99ee123d7b53 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:17:36.306388 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414cb3b8-5f37-4c3d-a38a-a8d89a10774b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:17:36.307625 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414cb3b8-5f37-4c3d-a38a-a8d89a10774b admin admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:17:36.307961 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414cb3b8-5f37-4c3d-a38a-a8d89a10774b admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:17:36.308244 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414cb3b8-5f37-4c3d-a38a-a8d89a10774b admin admin] PATH_INFO: `/v3/users` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:17:36.309045 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414cb3b8-5f37-4c3d-a38a-a8d89a10774b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:17:36.311315 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414cb3b8-5f37-4c3d-a38a-a8d89a10774b admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:17:36.316665 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 797/1584] 10.222.0.26 () {60 vars in 1143 bytes} [Thu Jul 14 09:17:36 2022] POST /identity/v3/users => generated 293 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 09:17:36.316665 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e625f2-1837-4e46-a87d-12dea996a2e7 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:17:36.325116 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e625f2-1837-4e46-a87d-12dea996a2e7 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:17:36.330227 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d64b244-2417-4c2c-bba0-c1138ec8af15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:17:36.330968 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d64b244-2417-4c2c-bba0-c1138ec8af15 admin admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:17:36.331362 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d64b244-2417-4c2c-bba0-c1138ec8af15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:17:36.331708 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d64b244-2417-4c2c-bba0-c1138ec8af15 admin admin] PATH_INFO: `/v3/users` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:17:36.332468 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d64b244-2417-4c2c-bba0-c1138ec8af15 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:17:36.334277 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 788/1585] 10.222.0.26 () {60 vars in 1143 bytes} [Thu Jul 14 09:17:36 2022] POST /identity/v3/users => generated 287 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 09:17:36.334534 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d64b244-2417-4c2c-bba0-c1138ec8af15 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:17:36.334955 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414cb3b8-5f37-4c3d-a38a-a8d89a10774b admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:17:36.344816 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414cb3b8-5f37-4c3d-a38a-a8d89a10774b admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:17:36.349704 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c88133-3526-4447-b3da-f5c496a169df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:17:36.350355 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c88133-3526-4447-b3da-f5c496a169df admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:17:36.350639 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c88133-3526-4447-b3da-f5c496a169df admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:17:36.350921 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c88133-3526-4447-b3da-f5c496a169df admin admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:17:36.351677 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c88133-3526-4447-b3da-f5c496a169df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:17:36.353275 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c88133-3526-4447-b3da-f5c496a169df admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:17:36.358336 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 798/1586] 10.222.0.26 () {60 vars in 1143 bytes} [Thu Jul 14 09:17:36 2022] POST /identity/v3/users => generated 286 bytes in 41 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 09:17:36.358336 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d64b244-2417-4c2c-bba0-c1138ec8af15 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:17:36.364246 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 789/1587] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jul 14 09:17:36 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 09:17:36.364716 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c88133-3526-4447-b3da-f5c496a169df admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:17:36.367873 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d64b244-2417-4c2c-bba0-c1138ec8af15 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:17:36.372594 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0c4299-e2a4-407b-a6d4-5cafe00421de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:17:36.373251 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0c4299-e2a4-407b-a6d4-5cafe00421de admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:17:36.373536 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0c4299-e2a4-407b-a6d4-5cafe00421de admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:17:36.373815 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0c4299-e2a4-407b-a6d4-5cafe00421de admin admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:17:36.374350 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c88133-3526-4447-b3da-f5c496a169df admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:17:36.374578 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0c4299-e2a4-407b-a6d4-5cafe00421de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:17:36.376239 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0c4299-e2a4-407b-a6d4-5cafe00421de admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:17:36.378939 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3050fad2-4e14-48ae-852d-47369804e91f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:17:36.379654 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3050fad2-4e14-48ae-852d-47369804e91f admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:17:36.379940 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3050fad2-4e14-48ae-852d-47369804e91f admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:17:36.380211 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3050fad2-4e14-48ae-852d-47369804e91f admin admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:17:36.380942 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3050fad2-4e14-48ae-852d-47369804e91f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:17:36.382539 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3050fad2-4e14-48ae-852d-47369804e91f admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:17:36.387626 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 799/1588] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jul 14 09:17:36 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 09:17:36.388435 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0c4299-e2a4-407b-a6d4-5cafe00421de admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:17:36.396332 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 790/1589] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jul 14 09:17:36 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 09:17:36.397030 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3050fad2-4e14-48ae-852d-47369804e91f admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:17:36.398877 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0c4299-e2a4-407b-a6d4-5cafe00421de admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:17:36.403659 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08396f9-0b0b-4bd3-ae2b-5af072d7e7ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:17:36.404535 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08396f9-0b0b-4bd3-ae2b-5af072d7e7ca admin admin] REQUEST_METHOD: `PUT` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:17:36.404816 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08396f9-0b0b-4bd3-ae2b-5af072d7e7ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:17:36.405251 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08396f9-0b0b-4bd3-ae2b-5af072d7e7ca admin admin] PATH_INFO: `/v3/projects/f835351847a8450c9344f052a327cf67/users/903e43d8e5fa473c9042bf8eeb99b2ba/roles/d1691322ce6a4edab7ce722edf0f2988` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:17:36.409292 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:17:36.420962 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08396f9-0b0b-4bd3-ae2b-5af072d7e7ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=f835351847a8450c9344f052a327cf67, user_id=903e43d8e5fa473c9042bf8eeb99b2ba, role_id=d1691322ce6a4edab7ce722edf0f2988)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:17:36.422133 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: WARNING py.warnings [None req-d08396f9-0b0b-4bd3-ae2b-5af072d7e7ca admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 14 09:17:36.422133 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: warnings.warn(msg) Jul 14 09:17:36.422133 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: Jul 14 09:17:36.424885 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08396f9-0b0b-4bd3-ae2b-5af072d7e7ca admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:17:36.439311 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 800/1590] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jul 14 09:17:36 2022] PUT /identity/v3/projects/f835351847a8450c9344f052a327cf67/users/903e43d8e5fa473c9042bf8eeb99b2ba/roles/d1691322ce6a4edab7ce722edf0f2988 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:17:36.440237 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08396f9-0b0b-4bd3-ae2b-5af072d7e7ca admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:17:36.454013 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3050fad2-4e14-48ae-852d-47369804e91f admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:17:36.501981 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66aeaa9-c198-418c-b366-82275165fccc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '952911c492444ffea4b211a24b564117', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:17:36.502633 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66aeaa9-c198-418c-b366-82275165fccc service neutron] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:17:36.502912 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66aeaa9-c198-418c-b366-82275165fccc service neutron] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:17:36.503184 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66aeaa9-c198-418c-b366-82275165fccc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:17:36.507310 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08396f9-0b0b-4bd3-ae2b-5af072d7e7ca admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:17:36.512419 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66aeaa9-c198-418c-b366-82275165fccc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:17:36.512566 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8ab4dc-56d1-4a7a-b339-4b42e908533b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:17:36.513114 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8ab4dc-56d1-4a7a-b339-4b42e908533b admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:17:36.513396 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8ab4dc-56d1-4a7a-b339-4b42e908533b admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:17:36.513700 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8ab4dc-56d1-4a7a-b339-4b42e908533b admin admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:17:36.513874 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66aeaa9-c198-418c-b366-82275165fccc service neutron] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:17:36.514436 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8ab4dc-56d1-4a7a-b339-4b42e908533b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:17:36.516073 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8ab4dc-56d1-4a7a-b339-4b42e908533b admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:17:36.524310 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 791/1591] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:17:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:17:36.525103 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66aeaa9-c198-418c-b366-82275165fccc service neutron] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:17:36.529479 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 801/1592] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jul 14 09:17:36 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 09:17:36.529479 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8ab4dc-56d1-4a7a-b339-4b42e908533b admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:17:36.535053 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66aeaa9-c198-418c-b366-82275165fccc service neutron] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:17:36.540575 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c319332-9a78-439d-92c1-df94050fd1b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:17:36.541248 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c319332-9a78-439d-92c1-df94050fd1b5 admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:17:36.541612 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c319332-9a78-439d-92c1-df94050fd1b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:17:36.541894 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c319332-9a78-439d-92c1-df94050fd1b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:17:36.542667 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c319332-9a78-439d-92c1-df94050fd1b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:17:36.543067 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8ab4dc-56d1-4a7a-b339-4b42e908533b admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:17:36.544328 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c319332-9a78-439d-92c1-df94050fd1b5 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:17:36.550698 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bd3caf-cdd8-4722-b222-8b864b80f0e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:17:36.551395 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bd3caf-cdd8-4722-b222-8b864b80f0e1 admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:17:36.551673 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bd3caf-cdd8-4722-b222-8b864b80f0e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:17:36.552018 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bd3caf-cdd8-4722-b222-8b864b80f0e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:17:36.552667 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bd3caf-cdd8-4722-b222-8b864b80f0e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:17:36.554277 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bd3caf-cdd8-4722-b222-8b864b80f0e1 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:17:36.555915 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 792/1593] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jul 14 09:17:36 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 09:17:36.556657 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c319332-9a78-439d-92c1-df94050fd1b5 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:17:36.568868 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 802/1594] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jul 14 09:17:36 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 09:17:36.569722 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bd3caf-cdd8-4722-b222-8b864b80f0e1 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:17:36.572399 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c319332-9a78-439d-92c1-df94050fd1b5 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:17:36.577846 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f84b47-6092-4d83-b389-e8886fb8c620 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:17:36.578522 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f84b47-6092-4d83-b389-e8886fb8c620 admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:17:36.578802 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f84b47-6092-4d83-b389-e8886fb8c620 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:17:36.579076 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f84b47-6092-4d83-b389-e8886fb8c620 admin admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:17:36.579844 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f84b47-6092-4d83-b389-e8886fb8c620 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:17:36.580488 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bd3caf-cdd8-4722-b222-8b864b80f0e1 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:17:36.581555 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f84b47-6092-4d83-b389-e8886fb8c620 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:17:36.588948 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f9c021-b331-41f3-a747-8c233bc6c488 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:17:36.589319 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f9c021-b331-41f3-a747-8c233bc6c488 admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:17:36.589596 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f9c021-b331-41f3-a747-8c233bc6c488 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:17:36.589892 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f9c021-b331-41f3-a747-8c233bc6c488 admin admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:17:36.590588 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f9c021-b331-41f3-a747-8c233bc6c488 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:17:36.592218 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f9c021-b331-41f3-a747-8c233bc6c488 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:17:36.596129 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 793/1595] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jul 14 09:17:36 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 09:17:36.596129 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f84b47-6092-4d83-b389-e8886fb8c620 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:17:36.604220 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f84b47-6092-4d83-b389-e8886fb8c620 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:17:36.607486 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 803/1596] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jul 14 09:17:36 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 09:17:36.611133 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169c3634-3a6b-4340-80df-a067fa297186 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:17:36.611133 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169c3634-3a6b-4340-80df-a067fa297186 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:17:36.611133 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169c3634-3a6b-4340-80df-a067fa297186 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:17:36.614402 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61748533-750f-4e90-9e10-400e7cef3b1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:17:36.615455 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61748533-750f-4e90-9e10-400e7cef3b1c admin admin] REQUEST_METHOD: `PUT` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:17:36.615735 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61748533-750f-4e90-9e10-400e7cef3b1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:17:36.616864 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61748533-750f-4e90-9e10-400e7cef3b1c admin admin] PATH_INFO: `/v3/projects/fe8ab23ae4884afc863a58c5e0b96e5b/users/18377cf2b2d14cb0b4cbda6ec720ed28/roles/d1691322ce6a4edab7ce722edf0f2988` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:17:36.635058 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61748533-750f-4e90-9e10-400e7cef3b1c admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe8ab23ae4884afc863a58c5e0b96e5b, user_id=18377cf2b2d14cb0b4cbda6ec720ed28, role_id=d1691322ce6a4edab7ce722edf0f2988)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:17:36.638307 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61748533-750f-4e90-9e10-400e7cef3b1c admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:17:36.638734 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-169c3634-3a6b-4340-80df-a067fa297186 None None] MFA Rules not processed for user `903e43d8e5fa473c9042bf8eeb99b2ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:17:36.639998 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-169c3634-3a6b-4340-80df-a067fa297186 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:17:36.658709 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 794/1597] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jul 14 09:17:36 2022] PUT /identity/v3/projects/fe8ab23ae4884afc863a58c5e0b96e5b/users/18377cf2b2d14cb0b4cbda6ec720ed28/roles/d1691322ce6a4edab7ce722edf0f2988 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:17:36.665923 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61748533-750f-4e90-9e10-400e7cef3b1c admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:17:36.747522 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 804/1598] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:17:36 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:17:36.751591 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169c3634-3a6b-4340-80df-a067fa297186 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:17:36.757376 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61748533-750f-4e90-9e10-400e7cef3b1c admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:17:36.763361 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6032448-0d7f-45a4-8221-e4bb4a24a8cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:17:36.763799 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169c3634-3a6b-4340-80df-a067fa297186 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:17:36.764204 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6032448-0d7f-45a4-8221-e4bb4a24a8cf admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:17:36.764699 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6032448-0d7f-45a4-8221-e4bb4a24a8cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:17:36.764834 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6032448-0d7f-45a4-8221-e4bb4a24a8cf admin admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:17:36.765660 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6032448-0d7f-45a4-8221-e4bb4a24a8cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:17:36.768736 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6032448-0d7f-45a4-8221-e4bb4a24a8cf admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:17:36.770918 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56e14ca-caf9-4859-ba08-f09c22b203e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:17:36.771708 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56e14ca-caf9-4859-ba08-f09c22b203e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:17:36.773309 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56e14ca-caf9-4859-ba08-f09c22b203e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:17:36.773441 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56e14ca-caf9-4859-ba08-f09c22b203e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:17:36.774179 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56e14ca-caf9-4859-ba08-f09c22b203e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:17:36.775972 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56e14ca-caf9-4859-ba08-f09c22b203e9 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:17:36.777709 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:17:36.783618 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 795/1599] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jul 14 09:17:36 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 09:17:36.783618 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6032448-0d7f-45a4-8221-e4bb4a24a8cf admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:17:36.790417 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 805/1600] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jul 14 09:17:36 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 09:17:36.790995 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56e14ca-caf9-4859-ba08-f09c22b203e9 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:17:36.795341 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6032448-0d7f-45a4-8221-e4bb4a24a8cf admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:17:36.801114 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431cc37c-efd7-46d6-bd36-de07e2394783 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:17:36.802008 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431cc37c-efd7-46d6-bd36-de07e2394783 admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:17:36.803522 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431cc37c-efd7-46d6-bd36-de07e2394783 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:17:36.803969 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431cc37c-efd7-46d6-bd36-de07e2394783 admin admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:17:36.804068 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56e14ca-caf9-4859-ba08-f09c22b203e9 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:17:36.804609 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431cc37c-efd7-46d6-bd36-de07e2394783 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:17:36.806392 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431cc37c-efd7-46d6-bd36-de07e2394783 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:17:36.811374 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1001a9a0-d857-483a-88f7-5afd6ccfa164 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:17:36.812420 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1001a9a0-d857-483a-88f7-5afd6ccfa164 admin admin] REQUEST_METHOD: `PUT` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:17:36.812716 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1001a9a0-d857-483a-88f7-5afd6ccfa164 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:17:36.812993 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1001a9a0-d857-483a-88f7-5afd6ccfa164 admin admin] PATH_INFO: `/v3/projects/5fef7aa711754cff8e9e1b50665df7ae/users/41a1046e199343b0a1a727ce6025260a/roles/d1691322ce6a4edab7ce722edf0f2988` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:17:36.820688 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 796/1601] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jul 14 09:17:36 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 09:17:36.820688 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431cc37c-efd7-46d6-bd36-de07e2394783 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:17:36.831437 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431cc37c-efd7-46d6-bd36-de07e2394783 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:17:36.831637 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1001a9a0-d857-483a-88f7-5afd6ccfa164 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5fef7aa711754cff8e9e1b50665df7ae, user_id=41a1046e199343b0a1a727ce6025260a, role_id=d1691322ce6a4edab7ce722edf0f2988)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:17:36.834479 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1001a9a0-d857-483a-88f7-5afd6ccfa164 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:17:36.836365 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040adf5e-a533-4b7d-9279-2d422eb7fcea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:17:36.837364 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040adf5e-a533-4b7d-9279-2d422eb7fcea admin admin] REQUEST_METHOD: `PUT` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:17:36.837733 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040adf5e-a533-4b7d-9279-2d422eb7fcea admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:17:36.838474 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040adf5e-a533-4b7d-9279-2d422eb7fcea admin admin] PATH_INFO: `/v3/projects/836b616f05a9421fbc776498f4ce7bd0/users/854fc6f464c940a78f779438aa3ee8a9/roles/d1691322ce6a4edab7ce722edf0f2988` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:17:36.842465 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:17:36.852420 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 806/1602] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jul 14 09:17:36 2022] PUT /identity/v3/projects/5fef7aa711754cff8e9e1b50665df7ae/users/41a1046e199343b0a1a727ce6025260a/roles/d1691322ce6a4edab7ce722edf0f2988 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:17:36.853740 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b388c14-be6a-45ca-8bb1-ca42e8db7b90 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:17:36.854365 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b388c14-be6a-45ca-8bb1-ca42e8db7b90 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:17:36.854649 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b388c14-be6a-45ca-8bb1-ca42e8db7b90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:17:36.857498 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040adf5e-a533-4b7d-9279-2d422eb7fcea admin admin] RBAC: Authorizing `identity:create_grant(project_id=836b616f05a9421fbc776498f4ce7bd0, user_id=854fc6f464c940a78f779438aa3ee8a9, role_id=d1691322ce6a4edab7ce722edf0f2988)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:17:36.860457 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040adf5e-a533-4b7d-9279-2d422eb7fcea admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:17:36.873698 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-5b388c14-be6a-45ca-8bb1-ca42e8db7b90 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:17:36.874924 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-5b388c14-be6a-45ca-8bb1-ca42e8db7b90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:17:36.884354 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 797/1603] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jul 14 09:17:36 2022] PUT /identity/v3/projects/836b616f05a9421fbc776498f4ce7bd0/users/854fc6f464c940a78f779438aa3ee8a9/roles/d1691322ce6a4edab7ce722edf0f2988 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:17:36.886270 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040adf5e-a533-4b7d-9279-2d422eb7fcea admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:17:36.954944 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 807/1604] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:17:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:17:36.954944 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b388c14-be6a-45ca-8bb1-ca42e8db7b90 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:17:36.969262 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b388c14-be6a-45ca-8bb1-ca42e8db7b90 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:17:36.973864 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41c2446-a705-4aeb-84f5-93952f9da5b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:17:36.974766 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41c2446-a705-4aeb-84f5-93952f9da5b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:17:36.975048 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41c2446-a705-4aeb-84f5-93952f9da5b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:17:36.975352 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41c2446-a705-4aeb-84f5-93952f9da5b9 admin admin] PATH_INFO: `/v3/projects/9b8c0df760e04c5ca5f000c048f1da4c/users/a2ab766e96024b899a01c540d9bd1322/roles/d1691322ce6a4edab7ce722edf0f2988` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:17:36.978454 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040adf5e-a533-4b7d-9279-2d422eb7fcea admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:17:36.983667 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd66c42-8db5-4839-b680-b22475efb26f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:17:36.984456 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd66c42-8db5-4839-b680-b22475efb26f admin admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:17:36.984806 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd66c42-8db5-4839-b680-b22475efb26f admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:17:36.985154 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd66c42-8db5-4839-b680-b22475efb26f admin admin] PATH_INFO: `/v3/projects` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:17:36.985953 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd66c42-8db5-4839-b680-b22475efb26f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:17:36.988454 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd66c42-8db5-4839-b680-b22475efb26f admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:17:36.992586 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41c2446-a705-4aeb-84f5-93952f9da5b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b8c0df760e04c5ca5f000c048f1da4c, user_id=a2ab766e96024b899a01c540d9bd1322, role_id=d1691322ce6a4edab7ce722edf0f2988)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:17:36.995536 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41c2446-a705-4aeb-84f5-93952f9da5b9 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:17:37.011338 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 808/1605] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jul 14 09:17:36 2022] PUT /identity/v3/projects/9b8c0df760e04c5ca5f000c048f1da4c/users/a2ab766e96024b899a01c540d9bd1322/roles/d1691322ce6a4edab7ce722edf0f2988 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:17:37.012311 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41c2446-a705-4aeb-84f5-93952f9da5b9 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:17:37.019283 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 798/1606] 10.222.0.26 () {60 vars in 1152 bytes} [Thu Jul 14 09:17:36 2022] POST /identity/v3/projects => generated 377 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 09:17:37.019283 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd66c42-8db5-4839-b680-b22475efb26f admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:17:37.089032 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd66c42-8db5-4839-b680-b22475efb26f admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:17:37.089303 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41c2446-a705-4aeb-84f5-93952f9da5b9 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:17:37.093900 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1616065-577c-447f-8a8f-d4b9ba733f7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:17:37.094743 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0361cc0-e1a0-4f1f-8e7f-4874ed56601f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:17:37.094883 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1616065-577c-447f-8a8f-d4b9ba733f7e admin admin] REQUEST_METHOD: `PUT` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:17:37.095077 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1616065-577c-447f-8a8f-d4b9ba733f7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:17:37.095387 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1616065-577c-447f-8a8f-d4b9ba733f7e admin admin] PATH_INFO: `/v3/projects/8f640560967147e3b567d705fc77fce0/users/0f11adab6ceb41ca9fbc3f3f54e11ec5/roles/d1691322ce6a4edab7ce722edf0f2988` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:17:37.096478 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0361cc0-e1a0-4f1f-8e7f-4874ed56601f admin admin] REQUEST_METHOD: `PUT` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:17:37.096911 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0361cc0-e1a0-4f1f-8e7f-4874ed56601f admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:17:37.097373 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0361cc0-e1a0-4f1f-8e7f-4874ed56601f admin admin] PATH_INFO: `/v3/projects/a49a3c1ac05e4d79822a655a88277da0/users/436ff0103dcf401d9363b2accfe635cb/roles/d1691322ce6a4edab7ce722edf0f2988` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:17:37.111093 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1616065-577c-447f-8a8f-d4b9ba733f7e admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f640560967147e3b567d705fc77fce0, user_id=0f11adab6ceb41ca9fbc3f3f54e11ec5, role_id=d1691322ce6a4edab7ce722edf0f2988)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:17:37.113800 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1616065-577c-447f-8a8f-d4b9ba733f7e admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:17:37.114200 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0361cc0-e1a0-4f1f-8e7f-4874ed56601f admin admin] RBAC: Authorizing `identity:create_grant(project_id=a49a3c1ac05e4d79822a655a88277da0, user_id=436ff0103dcf401d9363b2accfe635cb, role_id=d1691322ce6a4edab7ce722edf0f2988)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:17:37.116894 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0361cc0-e1a0-4f1f-8e7f-4874ed56601f admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:17:37.141770 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 809/1607] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jul 14 09:17:37 2022] PUT /identity/v3/projects/8f640560967147e3b567d705fc77fce0/users/0f11adab6ceb41ca9fbc3f3f54e11ec5/roles/d1691322ce6a4edab7ce722edf0f2988 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:17:37.141770 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b280fcfc-a6be-44b4-b8db-ecefd397730f None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:17:37.141770 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b280fcfc-a6be-44b4-b8db-ecefd397730f None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:17:37.141770 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b280fcfc-a6be-44b4-b8db-ecefd397730f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:17:37.142271 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 799/1608] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jul 14 09:17:37 2022] PUT /identity/v3/projects/a49a3c1ac05e4d79822a655a88277da0/users/436ff0103dcf401d9363b2accfe635cb/roles/d1691322ce6a4edab7ce722edf0f2988 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:17:37.142271 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2080bf-b53e-402d-9120-ef536f911976 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:17:37.142271 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2080bf-b53e-402d-9120-ef536f911976 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:17:37.142271 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2080bf-b53e-402d-9120-ef536f911976 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:17:37.163248 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-3d2080bf-b53e-402d-9120-ef536f911976 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:17:37.164878 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-3d2080bf-b53e-402d-9120-ef536f911976 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:17:37.174849 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-b280fcfc-a6be-44b4-b8db-ecefd397730f None None] MFA Rules not processed for user `41a1046e199343b0a1a727ce6025260a`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:17:37.176435 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-b280fcfc-a6be-44b4-b8db-ecefd397730f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:17:37.204555 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:17:37.222734 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:17:37.238831 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 800/1609] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:17:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:17:37.239715 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2080bf-b53e-402d-9120-ef536f911976 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:17:37.277899 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 810/1610] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:17:37 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:17:37.279586 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae21606-a25c-4dd4-92b4-26f97a2b964a None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:17:37.279974 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae21606-a25c-4dd4-92b4-26f97a2b964a None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:17:37.280310 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae21606-a25c-4dd4-92b4-26f97a2b964a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:17:37.301768 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-3ae21606-a25c-4dd4-92b4-26f97a2b964a None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:17:37.303054 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-3ae21606-a25c-4dd4-92b4-26f97a2b964a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:17:37.311689 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 811/1611] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:17:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:17:37.314886 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae21606-a25c-4dd4-92b4-26f97a2b964a None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:17:37.325320 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae21606-a25c-4dd4-92b4-26f97a2b964a None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:17:37.328024 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2080bf-b53e-402d-9120-ef536f911976 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:17:37.331633 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1586c1c5-66db-4382-80b9-8801d14bb4a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:17:37.332488 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1586c1c5-66db-4382-80b9-8801d14bb4a9 admin admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:17:37.332819 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ecd723-435c-4000-8e4a-82f063130057 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '952911c492444ffea4b211a24b564117', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:17:37.332961 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1586c1c5-66db-4382-80b9-8801d14bb4a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:17:37.333329 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1586c1c5-66db-4382-80b9-8801d14bb4a9 admin admin] PATH_INFO: `/v3/users` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:17:37.333907 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ecd723-435c-4000-8e4a-82f063130057 service neutron] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:17:37.334257 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1586c1c5-66db-4382-80b9-8801d14bb4a9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:17:37.334538 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ecd723-435c-4000-8e4a-82f063130057 service neutron] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:17:37.334957 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ecd723-435c-4000-8e4a-82f063130057 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:17:37.337114 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1586c1c5-66db-4382-80b9-8801d14bb4a9 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:17:37.346403 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ecd723-435c-4000-8e4a-82f063130057 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:17:37.348405 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ecd723-435c-4000-8e4a-82f063130057 service neutron] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:17:37.360930 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 801/1612] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:17:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:17:37.362248 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98de4cf-a433-48bf-95ea-151dd46fb54e None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:17:37.362533 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98de4cf-a433-48bf-95ea-151dd46fb54e None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:17:37.362836 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98de4cf-a433-48bf-95ea-151dd46fb54e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:17:37.367042 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 812/1613] 10.222.0.26 () {60 vars in 1143 bytes} [Thu Jul 14 09:17:37 2022] POST /identity/v3/users => generated 295 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 09:17:37.369439 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49488f1c-1155-4c2b-a240-7973ad750212 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:17:37.370214 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49488f1c-1155-4c2b-a240-7973ad750212 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:17:37.370721 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49488f1c-1155-4c2b-a240-7973ad750212 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:17:37.396427 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-c98de4cf-a433-48bf-95ea-151dd46fb54e None None] MFA Rules not processed for user `0f11adab6ceb41ca9fbc3f3f54e11ec5`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:17:37.397858 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-c98de4cf-a433-48bf-95ea-151dd46fb54e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:17:37.414150 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-49488f1c-1155-4c2b-a240-7973ad750212 None None] MFA Rules not processed for user `436ff0103dcf401d9363b2accfe635cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:17:37.416855 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-49488f1c-1155-4c2b-a240-7973ad750212 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:17:37.505022 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 802/1614] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:17:37 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:17:37.506374 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98de4cf-a433-48bf-95ea-151dd46fb54e None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:17:37.524264 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98de4cf-a433-48bf-95ea-151dd46fb54e None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:17:37.536867 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27695da6-bab9-4002-ad2e-c7cd02ccc7f3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '952911c492444ffea4b211a24b564117', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:17:37.539540 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27695da6-bab9-4002-ad2e-c7cd02ccc7f3 service neutron] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:17:37.539751 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27695da6-bab9-4002-ad2e-c7cd02ccc7f3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:17:37.540824 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27695da6-bab9-4002-ad2e-c7cd02ccc7f3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:17:37.542666 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 813/1615] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:17:37 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:17:37.545343 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49488f1c-1155-4c2b-a240-7973ad750212 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:17:37.560363 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49488f1c-1155-4c2b-a240-7973ad750212 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:17:37.566862 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27695da6-bab9-4002-ad2e-c7cd02ccc7f3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:17:37.567070 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f0f506-3f27-4da0-9916-d25c9e8b7791 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:17:37.568106 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f0f506-3f27-4da0-9916-d25c9e8b7791 admin admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:17:37.568546 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f0f506-3f27-4da0-9916-d25c9e8b7791 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:17:37.568910 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f0f506-3f27-4da0-9916-d25c9e8b7791 admin admin] PATH_INFO: `/v3/projects` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:17:37.570343 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f0f506-3f27-4da0-9916-d25c9e8b7791 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:17:37.571764 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27695da6-bab9-4002-ad2e-c7cd02ccc7f3 service neutron] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:17:37.573574 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f0f506-3f27-4da0-9916-d25c9e8b7791 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:17:37.600886 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 803/1616] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:17:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:17:37.603050 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27695da6-bab9-4002-ad2e-c7cd02ccc7f3 service neutron] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:17:37.605301 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-27695da6-bab9-4002-ad2e-c7cd02ccc7f3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:17:37.608460 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 814/1617] 10.222.0.26 () {60 vars in 1152 bytes} [Thu Jul 14 09:17:37 2022] POST /identity/v3/projects => generated 381 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 09:17:37.608460 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f0f506-3f27-4da0-9916-d25c9e8b7791 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:17:37.622447 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27695da6-bab9-4002-ad2e-c7cd02ccc7f3 service neutron] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:17:37.689046 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f0f506-3f27-4da0-9916-d25c9e8b7791 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:17:37.699856 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2276be50-4357-420d-9daa-ebe9606edcbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:17:37.700519 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2300cca2-1b5b-4f59-8ef5-7d5426a50373 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '952911c492444ffea4b211a24b564117', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:17:37.701175 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2276be50-4357-420d-9daa-ebe9606edcbf admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:17:37.701576 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2276be50-4357-420d-9daa-ebe9606edcbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:17:37.701841 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2300cca2-1b5b-4f59-8ef5-7d5426a50373 service neutron] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:17:37.701995 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2276be50-4357-420d-9daa-ebe9606edcbf admin admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:17:37.702509 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2300cca2-1b5b-4f59-8ef5-7d5426a50373 service neutron] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:17:37.702966 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2276be50-4357-420d-9daa-ebe9606edcbf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:17:37.703178 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2300cca2-1b5b-4f59-8ef5-7d5426a50373 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:17:37.705687 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2276be50-4357-420d-9daa-ebe9606edcbf admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:17:37.713535 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:17:37.715041 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2300cca2-1b5b-4f59-8ef5-7d5426a50373 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:17:37.717829 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2300cca2-1b5b-4f59-8ef5-7d5426a50373 service neutron] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:17:37.724777 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 815/1618] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jul 14 09:17:37 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 09:17:37.727406 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8590093c-c968-4dc4-9911-f7e5b1969ba2 None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:17:37.727708 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8590093c-c968-4dc4-9911-f7e5b1969ba2 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:17:37.727975 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8590093c-c968-4dc4-9911-f7e5b1969ba2 None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:17:37.732676 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 804/1619] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:17:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:17:37.733879 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2300cca2-1b5b-4f59-8ef5-7d5426a50373 service neutron] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:17:37.735382 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 816/1620] 10.222.0.26 () {54 vars in 961 bytes} [Thu Jul 14 09:17:37 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:17:37.736280 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8590093c-c968-4dc4-9911-f7e5b1969ba2 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:17:37.747620 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2300cca2-1b5b-4f59-8ef5-7d5426a50373 service neutron] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:17:37.754327 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8590093c-c968-4dc4-9911-f7e5b1969ba2 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:17:37.754799 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667f4a4b-a2bd-41f7-b8d9-07c74b9c1d3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:17:37.755972 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667f4a4b-a2bd-41f7-b8d9-07c74b9c1d3c admin admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:17:37.756847 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667f4a4b-a2bd-41f7-b8d9-07c74b9c1d3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:17:37.757548 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667f4a4b-a2bd-41f7-b8d9-07c74b9c1d3c admin admin] PATH_INFO: `/v3/projects` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:17:37.758775 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667f4a4b-a2bd-41f7-b8d9-07c74b9c1d3c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:17:37.761921 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c08584-1595-4d5e-9924-4e89298ef61d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:17:37.762793 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c08584-1595-4d5e-9924-4e89298ef61d admin admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:17:37.762996 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667f4a4b-a2bd-41f7-b8d9-07c74b9c1d3c admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:17:37.763260 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c08584-1595-4d5e-9924-4e89298ef61d admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:17:37.763677 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c08584-1595-4d5e-9924-4e89298ef61d admin admin] PATH_INFO: `/v3/users` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:17:37.764780 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c08584-1595-4d5e-9924-4e89298ef61d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:17:37.769168 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c08584-1595-4d5e-9924-4e89298ef61d admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:17:37.784867 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:17:37.807323 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 817/1621] 10.222.0.26 () {60 vars in 1143 bytes} [Thu Jul 14 09:17:37 2022] POST /identity/v3/users => generated 297 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 09:17:37.808295 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c08584-1595-4d5e-9924-4e89298ef61d admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:17:37.818913 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 805/1622] 10.222.0.26 () {60 vars in 1152 bytes} [Thu Jul 14 09:17:37 2022] POST /identity/v3/projects => generated 369 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 09:17:37.820443 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab09130c-2109-4994-a9aa-834e905efcb3 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:17:37.823055 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab09130c-2109-4994-a9aa-834e905efcb3 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:17:37.823984 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab09130c-2109-4994-a9aa-834e905efcb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:17:37.838474 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c08584-1595-4d5e-9924-4e89298ef61d admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:17:37.952268 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-ab09130c-2109-4994-a9aa-834e905efcb3 None None] MFA Rules not processed for user `538fff3c5a2b487eb44ae7c1a1198102`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:17:37.956791 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-ab09130c-2109-4994-a9aa-834e905efcb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:17:38.004266 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00500b52-8a4f-46ef-90b9-ae9eb3612627 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:17:38.005884 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00500b52-8a4f-46ef-90b9-ae9eb3612627 admin admin] REQUEST_METHOD: `PUT` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:17:38.007712 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00500b52-8a4f-46ef-90b9-ae9eb3612627 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:17:38.008241 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00500b52-8a4f-46ef-90b9-ae9eb3612627 admin admin] PATH_INFO: `/v3/projects/16d8a1d9b6254a9e9f44bda04b2880c3/users/03351ab734a143748d707a1b4650e90e/roles/7b549d58d1724395b343609d895c80d0` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:17:38.049484 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00500b52-8a4f-46ef-90b9-ae9eb3612627 admin admin] RBAC: Authorizing `identity:create_grant(project_id=16d8a1d9b6254a9e9f44bda04b2880c3, user_id=03351ab734a143748d707a1b4650e90e, role_id=7b549d58d1724395b343609d895c80d0)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:17:38.052743 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00500b52-8a4f-46ef-90b9-ae9eb3612627 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:17:38.059698 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 806/1623] 10.222.0.26 () {58 vars in 1058 bytes} [Thu Jul 14 09:17:37 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:17:38.061355 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab09130c-2109-4994-a9aa-834e905efcb3 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:17:38.076570 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab09130c-2109-4994-a9aa-834e905efcb3 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:17:38.081381 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 818/1624] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jul 14 09:17:37 2022] PUT /identity/v3/projects/16d8a1d9b6254a9e9f44bda04b2880c3/users/03351ab734a143748d707a1b4650e90e/roles/7b549d58d1724395b343609d895c80d0 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:17:38.087328 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00500b52-8a4f-46ef-90b9-ae9eb3612627 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:17:38.180156 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30374918-67fc-4b38-8edc-83c7342988ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:17:38.181531 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30374918-67fc-4b38-8edc-83c7342988ff admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:17:38.182325 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30374918-67fc-4b38-8edc-83c7342988ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:17:38.182888 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30374918-67fc-4b38-8edc-83c7342988ff admin admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:17:38.184796 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30374918-67fc-4b38-8edc-83c7342988ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:17:38.187048 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30374918-67fc-4b38-8edc-83c7342988ff admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:17:38.205147 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 807/1625] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jul 14 09:17:38 2022] GET /identity/v3/roles => generated 1420 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 09:17:38.216715 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30374918-67fc-4b38-8edc-83c7342988ff admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:17:38.274278 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00500b52-8a4f-46ef-90b9-ae9eb3612627 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:17:38.287556 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9974eee5-f14d-4e66-b0ab-a6d99ee5c942 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:17:38.288365 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9974eee5-f14d-4e66-b0ab-a6d99ee5c942 admin admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:17:38.291293 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9974eee5-f14d-4e66-b0ab-a6d99ee5c942 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:17:38.291863 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9974eee5-f14d-4e66-b0ab-a6d99ee5c942 admin admin] PATH_INFO: `/v3/users` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:17:38.292681 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9974eee5-f14d-4e66-b0ab-a6d99ee5c942 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:17:38.297077 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9974eee5-f14d-4e66-b0ab-a6d99ee5c942 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:17:38.349682 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 819/1626] 10.222.0.26 () {60 vars in 1143 bytes} [Thu Jul 14 09:17:38 2022] POST /identity/v3/users => generated 291 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 09:17:38.356389 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9974eee5-f14d-4e66-b0ab-a6d99ee5c942 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:17:38.369208 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30374918-67fc-4b38-8edc-83c7342988ff admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:17:38.380104 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f385e75d-cfb7-4684-8951-e6557fe95b9f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '538fff3c5a2b487eb44ae7c1a1198102', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:17:38.381130 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f385e75d-cfb7-4684-8951-e6557fe95b9f service nova] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:17:38.381356 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f385e75d-cfb7-4684-8951-e6557fe95b9f service nova] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:17:38.381875 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f385e75d-cfb7-4684-8951-e6557fe95b9f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:17:38.382290 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9974eee5-f14d-4e66-b0ab-a6d99ee5c942 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:17:38.389064 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:17:38.390965 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d39f3e5-3574-4e35-a387-2e7afdb53c1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:17:38.392232 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d39f3e5-3574-4e35-a387-2e7afdb53c1d admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:17:38.392670 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d39f3e5-3574-4e35-a387-2e7afdb53c1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:17:38.393035 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d39f3e5-3574-4e35-a387-2e7afdb53c1d admin admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:17:38.394124 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d39f3e5-3574-4e35-a387-2e7afdb53c1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:17:38.396096 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d39f3e5-3574-4e35-a387-2e7afdb53c1d admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:17:38.415707 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 820/1627] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jul 14 09:17:38 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 09:17:38.417194 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d39f3e5-3574-4e35-a387-2e7afdb53c1d admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:17:38.434666 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d39f3e5-3574-4e35-a387-2e7afdb53c1d admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:17:38.441467 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6427e2-e6d1-4e4e-bf9f-e60d97869df3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:17:38.443534 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6427e2-e6d1-4e4e-bf9f-e60d97869df3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:17:38.445346 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6427e2-e6d1-4e4e-bf9f-e60d97869df3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:17:38.449279 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6427e2-e6d1-4e4e-bf9f-e60d97869df3 admin admin] PATH_INFO: `/v3/projects/c779db3b2b21484abc448a330bd36c79/users/96ea3b17dbea446bb961dcea712ad169/roles/7b549d58d1724395b343609d895c80d0` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:17:38.492367 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6427e2-e6d1-4e4e-bf9f-e60d97869df3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c779db3b2b21484abc448a330bd36c79, user_id=96ea3b17dbea446bb961dcea712ad169, role_id=7b549d58d1724395b343609d895c80d0)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:17:38.511126 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6427e2-e6d1-4e4e-bf9f-e60d97869df3 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:17:38.544452 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 821/1628] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jul 14 09:17:38 2022] PUT /identity/v3/projects/c779db3b2b21484abc448a330bd36c79/users/96ea3b17dbea446bb961dcea712ad169/roles/7b549d58d1724395b343609d895c80d0 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:17:38.550337 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f385e75d-cfb7-4684-8951-e6557fe95b9f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:17:38.552263 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6427e2-e6d1-4e4e-bf9f-e60d97869df3 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:17:38.563039 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f385e75d-cfb7-4684-8951-e6557fe95b9f service nova] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:17:38.582453 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 808/1629] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:17:38 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:17:38.583470 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f385e75d-cfb7-4684-8951-e6557fe95b9f service nova] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:17:38.593247 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:17:38.605358 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f385e75d-cfb7-4684-8951-e6557fe95b9f service nova] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:17:38.670329 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6427e2-e6d1-4e4e-bf9f-e60d97869df3 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:17:38.684156 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47960dc-f7c9-467b-9aa3-a80bf14705e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:17:38.685032 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47960dc-f7c9-467b-9aa3-a80bf14705e4 admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:17:38.685252 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47960dc-f7c9-467b-9aa3-a80bf14705e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:17:38.685536 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47960dc-f7c9-467b-9aa3-a80bf14705e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:17:38.687004 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47960dc-f7c9-467b-9aa3-a80bf14705e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:17:38.688716 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47960dc-f7c9-467b-9aa3-a80bf14705e4 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:17:38.694898 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660d565b-4adf-4954-b691-7499eea3350b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:17:38.696615 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660d565b-4adf-4954-b691-7499eea3350b admin admin] REQUEST_METHOD: `PUT` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:17:38.697369 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660d565b-4adf-4954-b691-7499eea3350b admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:17:38.697369 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660d565b-4adf-4954-b691-7499eea3350b admin admin] PATH_INFO: `/v3/projects/16d8a1d9b6254a9e9f44bda04b2880c3/users/03351ab734a143748d707a1b4650e90e/roles/7b549d58d1724395b343609d895c80d0` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:17:38.699700 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660d565b-4adf-4954-b691-7499eea3350b admin admin] RBAC: Authorizing `identity:create_grant(project_id=16d8a1d9b6254a9e9f44bda04b2880c3, user_id=03351ab734a143748d707a1b4650e90e, role_id=7b549d58d1724395b343609d895c80d0)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:17:38.706067 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660d565b-4adf-4954-b691-7499eea3350b admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:17:38.714883 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 822/1630] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jul 14 09:17:38 2022] GET /identity/v3/roles => generated 1420 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 09:17:38.715958 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47960dc-f7c9-467b-9aa3-a80bf14705e4 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:17:38.723505 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 809/1631] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jul 14 09:17:38 2022] PUT /identity/v3/projects/16d8a1d9b6254a9e9f44bda04b2880c3/users/03351ab734a143748d707a1b4650e90e/roles/7b549d58d1724395b343609d895c80d0 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:17:38.734326 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660d565b-4adf-4954-b691-7499eea3350b admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:17:38.742854 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47960dc-f7c9-467b-9aa3-a80bf14705e4 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:17:38.849834 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660d565b-4adf-4954-b691-7499eea3350b admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:17:38.858933 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5bd5f9-9d8e-4e32-a2eb-a083131aaf36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:17:38.860300 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5bd5f9-9d8e-4e32-a2eb-a083131aaf36 admin admin] REQUEST_METHOD: `PUT` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:17:38.860806 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5bd5f9-9d8e-4e32-a2eb-a083131aaf36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:17:38.861337 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5bd5f9-9d8e-4e32-a2eb-a083131aaf36 admin admin] PATH_INFO: `/v3/projects/63ced92de28b475a9ab77520514a4c37/users/3f0a6ce265fd4b608ac5c264c89a9fd4/roles/7b549d58d1724395b343609d895c80d0` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:17:38.936983 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5bd5f9-9d8e-4e32-a2eb-a083131aaf36 admin admin] RBAC: Authorizing `identity:create_grant(project_id=63ced92de28b475a9ab77520514a4c37, user_id=3f0a6ce265fd4b608ac5c264c89a9fd4, role_id=7b549d58d1724395b343609d895c80d0)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:17:38.936983 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5bd5f9-9d8e-4e32-a2eb-a083131aaf36 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:17:38.938461 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45797509-c4ff-4f0c-891c-9b6cb9d5aa25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:17:38.938461 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45797509-c4ff-4f0c-891c-9b6cb9d5aa25 admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:17:38.938461 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45797509-c4ff-4f0c-891c-9b6cb9d5aa25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:17:38.938461 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45797509-c4ff-4f0c-891c-9b6cb9d5aa25 admin admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:17:38.940115 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45797509-c4ff-4f0c-891c-9b6cb9d5aa25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:17:38.940115 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45797509-c4ff-4f0c-891c-9b6cb9d5aa25 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:17:38.940566 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 810/1632] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jul 14 09:17:38 2022] PUT /identity/v3/projects/63ced92de28b475a9ab77520514a4c37/users/3f0a6ce265fd4b608ac5c264c89a9fd4/roles/7b549d58d1724395b343609d895c80d0 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:17:38.940566 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf5d7af-1728-479c-b8c6-e36bfc96123c None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:17:38.940566 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf5d7af-1728-479c-b8c6-e36bfc96123c None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:17:38.940566 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf5d7af-1728-479c-b8c6-e36bfc96123c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:17:38.951132 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 823/1633] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jul 14 09:17:38 2022] GET /identity/v3/roles => generated 1420 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 09:17:38.953805 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45797509-c4ff-4f0c-891c-9b6cb9d5aa25 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:17:39.004459 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-dbf5d7af-1728-479c-b8c6-e36bfc96123c None None] MFA Rules not processed for user `03351ab734a143748d707a1b4650e90e`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:17:39.006007 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-dbf5d7af-1728-479c-b8c6-e36bfc96123c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:17:39.008100 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45797509-c4ff-4f0c-891c-9b6cb9d5aa25 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:17:39.019618 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:17:39.134388 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd661c58-5371-432c-94e7-ce71766b36bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:17:39.137196 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd661c58-5371-432c-94e7-ce71766b36bb admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:17:39.137684 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd661c58-5371-432c-94e7-ce71766b36bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:17:39.138116 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd661c58-5371-432c-94e7-ce71766b36bb admin admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:17:39.139050 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd661c58-5371-432c-94e7-ce71766b36bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:17:39.141946 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd661c58-5371-432c-94e7-ce71766b36bb admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:17:39.146889 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 811/1634] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:17:38 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:17:39.148425 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf5d7af-1728-479c-b8c6-e36bfc96123c None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:17:39.168587 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 824/1635] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jul 14 09:17:38 2022] GET /identity/v3/roles => generated 1420 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 09:17:39.169586 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd661c58-5371-432c-94e7-ce71766b36bb admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:17:39.177120 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf5d7af-1728-479c-b8c6-e36bfc96123c None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:17:39.191832 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e73bd75-1521-4396-9565-93b8ebf53144 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:17:39.202539 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e73bd75-1521-4396-9565-93b8ebf53144 admin admin] REQUEST_METHOD: `PUT` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:17:39.202833 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e73bd75-1521-4396-9565-93b8ebf53144 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:17:39.203120 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e73bd75-1521-4396-9565-93b8ebf53144 admin admin] PATH_INFO: `/v3/projects/c779db3b2b21484abc448a330bd36c79/users/96ea3b17dbea446bb961dcea712ad169/roles/7b549d58d1724395b343609d895c80d0` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:17:39.218135 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e73bd75-1521-4396-9565-93b8ebf53144 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c779db3b2b21484abc448a330bd36c79, user_id=96ea3b17dbea446bb961dcea712ad169, role_id=7b549d58d1724395b343609d895c80d0)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:17:39.228981 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e73bd75-1521-4396-9565-93b8ebf53144 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:17:39.273099 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 812/1636] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jul 14 09:17:39 2022] PUT /identity/v3/projects/c779db3b2b21484abc448a330bd36c79/users/96ea3b17dbea446bb961dcea712ad169/roles/7b549d58d1724395b343609d895c80d0 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:17:39.274107 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e73bd75-1521-4396-9565-93b8ebf53144 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:17:39.376827 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:17:39.386768 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd661c58-5371-432c-94e7-ce71766b36bb admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:17:39.477294 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e73bd75-1521-4396-9565-93b8ebf53144 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:17:39.490937 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f095029b-a269-4e68-baed-0a4485158559 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:17:39.492044 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f095029b-a269-4e68-baed-0a4485158559 admin admin] REQUEST_METHOD: `PUT` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:17:39.492329 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f095029b-a269-4e68-baed-0a4485158559 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:17:39.492605 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f095029b-a269-4e68-baed-0a4485158559 admin admin] PATH_INFO: `/v3/projects/63ced92de28b475a9ab77520514a4c37/users/3f0a6ce265fd4b608ac5c264c89a9fd4/roles/7b549d58d1724395b343609d895c80d0` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:17:39.497813 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f095029b-a269-4e68-baed-0a4485158559 admin admin] RBAC: Authorizing `identity:create_grant(project_id=63ced92de28b475a9ab77520514a4c37, user_id=3f0a6ce265fd4b608ac5c264c89a9fd4, role_id=7b549d58d1724395b343609d895c80d0)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:17:39.501075 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f095029b-a269-4e68-baed-0a4485158559 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:17:39.506734 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a854e86-50a7-438b-89be-f73e4a4b4c14 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e76f1952a5b146808844b9f2b7395035', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:17:39.507233 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a854e86-50a7-438b-89be-f73e4a4b4c14 service cinder] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:17:39.507568 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a854e86-50a7-438b-89be-f73e4a4b4c14 service cinder] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:17:39.507919 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a854e86-50a7-438b-89be-f73e4a4b4c14 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:17:39.526300 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 813/1637] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jul 14 09:17:39 2022] PUT /identity/v3/projects/63ced92de28b475a9ab77520514a4c37/users/3f0a6ce265fd4b608ac5c264c89a9fd4/roles/7b549d58d1724395b343609d895c80d0 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:17:39.537183 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f63098-9aa7-4575-ad5b-e29b41b75406 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:17:39.537183 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f63098-9aa7-4575-ad5b-e29b41b75406 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:17:39.537492 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f63098-9aa7-4575-ad5b-e29b41b75406 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:17:39.581886 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-13f63098-9aa7-4575-ad5b-e29b41b75406 None None] MFA Rules not processed for user `96ea3b17dbea446bb961dcea712ad169`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:17:39.583792 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-13f63098-9aa7-4575-ad5b-e29b41b75406 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:17:39.633065 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a854e86-50a7-438b-89be-f73e4a4b4c14 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:17:39.637468 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a854e86-50a7-438b-89be-f73e4a4b4c14 service cinder] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:17:39.684098 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 825/1638] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:17:39 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 514 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:17:39.687099 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfbb4bd-0f4d-46b8-b9ed-35ace8f3f875 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:17:39.688253 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfbb4bd-0f4d-46b8-b9ed-35ace8f3f875 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:17:39.688624 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfbb4bd-0f4d-46b8-b9ed-35ace8f3f875 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:17:39.747761 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 814/1639] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:17:39 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:17:39.752436 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-0bfbb4bd-0f4d-46b8-b9ed-35ace8f3f875 None None] MFA Rules not processed for user `3f0a6ce265fd4b608ac5c264c89a9fd4`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:17:39.753853 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-0bfbb4bd-0f4d-46b8-b9ed-35ace8f3f875 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:17:39.770370 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f63098-9aa7-4575-ad5b-e29b41b75406 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:17:39.807002 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:17:39.915960 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 826/1640] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:17:39 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:17:39.919746 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f63098-9aa7-4575-ad5b-e29b41b75406 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:17:39.920838 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2e272f-b013-45b8-9a00-378369045dde None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:17:39.921162 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2e272f-b013-45b8-9a00-378369045dde None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:17:39.921452 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2e272f-b013-45b8-9a00-378369045dde None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:17:39.923432 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 827/1641] 10.222.0.26 () {54 vars in 961 bytes} [Thu Jul 14 09:17:39 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:17:39.930242 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85afcd93-540a-4417-b445-cd0c1b1c21f9 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:17:39.930941 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85afcd93-540a-4417-b445-cd0c1b1c21f9 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:17:39.931153 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85afcd93-540a-4417-b445-cd0c1b1c21f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:17:39.936380 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b965d923-6e5a-41f6-8a92-19949a218a63 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e76f1952a5b146808844b9f2b7395035', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:17:39.937043 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b965d923-6e5a-41f6-8a92-19949a218a63 service cinder] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:17:39.937319 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b965d923-6e5a-41f6-8a92-19949a218a63 service cinder] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:17:39.937599 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b965d923-6e5a-41f6-8a92-19949a218a63 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:17:39.964306 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b965d923-6e5a-41f6-8a92-19949a218a63 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:17:39.965774 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b965d923-6e5a-41f6-8a92-19949a218a63 service cinder] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:17:39.969945 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-85afcd93-540a-4417-b445-cd0c1b1c21f9 None None] MFA Rules not processed for user `538fff3c5a2b487eb44ae7c1a1198102`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:17:39.971883 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-85afcd93-540a-4417-b445-cd0c1b1c21f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:17:39.991714 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 815/1642] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:17:39 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:17:40.038893 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b965d923-6e5a-41f6-8a92-19949a218a63 service cinder] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:17:40.050702 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b965d923-6e5a-41f6-8a92-19949a218a63 service cinder] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:17:40.062787 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6f5b84-9ff3-4c13-9e69-35b8aab788cb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e76f1952a5b146808844b9f2b7395035', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:17:40.063510 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6f5b84-9ff3-4c13-9e69-35b8aab788cb service cinder] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:17:40.063890 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6f5b84-9ff3-4c13-9e69-35b8aab788cb service cinder] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:17:40.064655 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6f5b84-9ff3-4c13-9e69-35b8aab788cb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:17:40.069924 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:17:40.092020 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6f5b84-9ff3-4c13-9e69-35b8aab788cb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:17:40.092020 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6f5b84-9ff3-4c13-9e69-35b8aab788cb service cinder] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:17:40.112492 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 816/1643] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:17:40 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:17:40.133489 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 828/1644] 10.222.0.26 () {58 vars in 1058 bytes} [Thu Jul 14 09:17:39 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:17:40.144376 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633222b2-242f-4358-a07d-ff6e55fb187b None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:17:40.144693 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633222b2-242f-4358-a07d-ff6e55fb187b None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:17:40.144977 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633222b2-242f-4358-a07d-ff6e55fb187b None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:17:40.147139 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 817/1645] 10.222.0.26 () {54 vars in 912 bytes} [Thu Jul 14 09:17:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:17:40.153916 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82701fa4-c1d5-4e62-804c-ac86016289b3 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:17:40.154135 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82701fa4-c1d5-4e62-804c-ac86016289b3 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:17:40.154421 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82701fa4-c1d5-4e62-804c-ac86016289b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:17:40.183956 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-82701fa4-c1d5-4e62-804c-ac86016289b3 None None] MFA Rules not processed for user `538fff3c5a2b487eb44ae7c1a1198102`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:17:40.185269 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-82701fa4-c1d5-4e62-804c-ac86016289b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:17:40.200926 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 829/1646] 10.222.0.26 () {58 vars in 1009 bytes} [Thu Jul 14 09:17:40 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:17:40.346920 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633222b2-242f-4358-a07d-ff6e55fb187b None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:17:40.364366 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633222b2-242f-4358-a07d-ff6e55fb187b None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:17:40.372887 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a05183-3af4-4a11-9f01-f01e7cd3b2d6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e76f1952a5b146808844b9f2b7395035', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:17:40.374499 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a05183-3af4-4a11-9f01-f01e7cd3b2d6 service cinder] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:17:40.375698 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a05183-3af4-4a11-9f01-f01e7cd3b2d6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:17:40.376762 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a05183-3af4-4a11-9f01-f01e7cd3b2d6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:17:40.396365 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a05183-3af4-4a11-9f01-f01e7cd3b2d6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:17:40.398279 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a05183-3af4-4a11-9f01-f01e7cd3b2d6 service cinder] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:17:40.423797 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 818/1647] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:17:40 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:17:40.532015 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82701fa4-c1d5-4e62-804c-ac86016289b3 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:17:40.547618 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82701fa4-c1d5-4e62-804c-ac86016289b3 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:17:40.558096 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fec8027-f4f4-4f6a-a4ae-4d8f4d035991 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e76f1952a5b146808844b9f2b7395035', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:17:40.559011 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fec8027-f4f4-4f6a-a4ae-4d8f4d035991 service cinder] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:17:40.559508 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fec8027-f4f4-4f6a-a4ae-4d8f4d035991 service cinder] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:17:40.561758 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fec8027-f4f4-4f6a-a4ae-4d8f4d035991 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:17:40.570304 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1705b8-723f-4bc1-8629-1f9e516ac1c3 None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:17:40.570591 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1705b8-723f-4bc1-8629-1f9e516ac1c3 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:17:40.570869 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1705b8-723f-4bc1-8629-1f9e516ac1c3 None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:17:40.573012 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 819/1648] 10.222.0.26 () {54 vars in 912 bytes} [Thu Jul 14 09:17:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:17:40.581230 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e679e657-cb68-479b-8243-c75bcaa691cd None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:17:40.582149 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e679e657-cb68-479b-8243-c75bcaa691cd None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:17:40.582314 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e679e657-cb68-479b-8243-c75bcaa691cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:17:40.617152 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-e679e657-cb68-479b-8243-c75bcaa691cd None None] MFA Rules not processed for user `538fff3c5a2b487eb44ae7c1a1198102`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:17:40.619080 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-e679e657-cb68-479b-8243-c75bcaa691cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:17:40.643814 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 820/1649] 10.222.0.26 () {58 vars in 1009 bytes} [Thu Jul 14 09:17:40 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 09:17:40.767072 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fec8027-f4f4-4f6a-a4ae-4d8f4d035991 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:17:40.774686 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fec8027-f4f4-4f6a-a4ae-4d8f4d035991 service cinder] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:17:40.791269 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e679e657-cb68-479b-8243-c75bcaa691cd None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:17:40.803338 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e679e657-cb68-479b-8243-c75bcaa691cd None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:17:40.806516 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 830/1650] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:17:40 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 275 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:17:40.815582 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56557101-3536-423e-a5ed-73a5cedc42d3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e76f1952a5b146808844b9f2b7395035', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:17:40.816564 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56557101-3536-423e-a5ed-73a5cedc42d3 service cinder] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:17:40.816998 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56557101-3536-423e-a5ed-73a5cedc42d3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:17:40.818868 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56557101-3536-423e-a5ed-73a5cedc42d3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:17:40.849178 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56557101-3536-423e-a5ed-73a5cedc42d3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:17:40.850823 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56557101-3536-423e-a5ed-73a5cedc42d3 service cinder] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:17:40.894409 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 821/1651] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:17:40 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 09:17:41.089763 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fec8027-f4f4-4f6a-a4ae-4d8f4d035991 service cinder] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:17:41.104818 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fec8027-f4f4-4f6a-a4ae-4d8f4d035991 service cinder] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:17:41.109231 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8318f156-6228-4e7b-a06f-ce355ed19eed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e76f1952a5b146808844b9f2b7395035', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:17:41.110518 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8318f156-6228-4e7b-a06f-ce355ed19eed service cinder] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:17:41.111117 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8318f156-6228-4e7b-a06f-ce355ed19eed service cinder] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:17:41.111616 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8318f156-6228-4e7b-a06f-ce355ed19eed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:17:41.236297 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8318f156-6228-4e7b-a06f-ce355ed19eed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:17:41.237895 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8318f156-6228-4e7b-a06f-ce355ed19eed service cinder] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:17:41.267926 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 831/1652] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:17:41 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:17:41.386916 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56557101-3536-423e-a5ed-73a5cedc42d3 service cinder] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:17:41.552932 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56557101-3536-423e-a5ed-73a5cedc42d3 service cinder] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:17:41.562653 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6669816-98ca-4482-809e-4c3cbf24ea51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:17:41.564437 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6669816-98ca-4482-809e-4c3cbf24ea51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:17:41.564871 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6669816-98ca-4482-809e-4c3cbf24ea51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:17:41.565444 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6669816-98ca-4482-809e-4c3cbf24ea51 admin admin] PATH_INFO: `/v3/users/0f11adab6ceb41ca9fbc3f3f54e11ec5` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:17:41.577882 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6669816-98ca-4482-809e-4c3cbf24ea51 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0f11adab6ceb41ca9fbc3f3f54e11ec5)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:17:41.582789 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6669816-98ca-4482-809e-4c3cbf24ea51 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:17:41.686558 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-e6669816-98ca-4482-809e-4c3cbf24ea51 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f11adab6ceb41ca9fbc3f3f54e11ec5'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:17:41.706395 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-e6669816-98ca-4482-809e-4c3cbf24ea51 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f11adab6ceb41ca9fbc3f3f54e11ec5'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:17:41.707482 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-e6669816-98ca-4482-809e-4c3cbf24ea51 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f11adab6ceb41ca9fbc3f3f54e11ec5'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:17:41.708428 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-e6669816-98ca-4482-809e-4c3cbf24ea51 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f11adab6ceb41ca9fbc3f3f54e11ec5'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:17:41.721746 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-e6669816-98ca-4482-809e-4c3cbf24ea51 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f11adab6ceb41ca9fbc3f3f54e11ec5'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:17:41.735735 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:17:41.761597 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 822/1653] 10.222.0.26 () {58 vars in 1223 bytes} [Thu Jul 14 09:17:41 2022] DELETE /identity/v3/users/0f11adab6ceb41ca9fbc3f3f54e11ec5 => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:17:41.767012 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de94f291-74a2-4b32-b607-85de165a5e5c None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:17:41.767012 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de94f291-74a2-4b32-b607-85de165a5e5c None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:17:41.767299 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de94f291-74a2-4b32-b607-85de165a5e5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:17:41.810399 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-de94f291-74a2-4b32-b607-85de165a5e5c None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:17:41.813614 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-de94f291-74a2-4b32-b607-85de165a5e5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:17:41.969765 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 832/1654] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:17:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:17:42.104676 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6669816-98ca-4482-809e-4c3cbf24ea51 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:17:42.109334 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-e6669816-98ca-4482-809e-4c3cbf24ea51 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:17:42.239420 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6669816-98ca-4482-809e-4c3cbf24ea51 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:17:42.255844 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cf2b4a-f320-488d-a366-fed9f3b7baa2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '952911c492444ffea4b211a24b564117', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:17:42.257081 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cf2b4a-f320-488d-a366-fed9f3b7baa2 service neutron] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:17:42.257583 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cf2b4a-f320-488d-a366-fed9f3b7baa2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:17:42.258077 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cf2b4a-f320-488d-a366-fed9f3b7baa2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:17:42.281201 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cf2b4a-f320-488d-a366-fed9f3b7baa2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:17:42.283247 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cf2b4a-f320-488d-a366-fed9f3b7baa2 service neutron] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:17:42.301280 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 823/1655] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:17:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 09:17:43.556069 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de94f291-74a2-4b32-b607-85de165a5e5c None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:17:43.558386 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-de94f291-74a2-4b32-b607-85de165a5e5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:17:43.616797 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de94f291-74a2-4b32-b607-85de165a5e5c None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:17:43.643942 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162a5b98-a790-4116-b5b8-9f6a2d84b3e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:17:43.644996 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162a5b98-a790-4116-b5b8-9f6a2d84b3e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:17:43.645376 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162a5b98-a790-4116-b5b8-9f6a2d84b3e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:17:43.645769 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162a5b98-a790-4116-b5b8-9f6a2d84b3e6 admin admin] PATH_INFO: `/v3/projects/8f640560967147e3b567d705fc77fce0` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:17:43.647638 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162a5b98-a790-4116-b5b8-9f6a2d84b3e6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8f640560967147e3b567d705fc77fce0)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:17:43.654851 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162a5b98-a790-4116-b5b8-9f6a2d84b3e6 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:17:43.663437 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:17:43.848754 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-162a5b98-a790-4116-b5b8-9f6a2d84b3e6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8f640560967147e3b567d705fc77fce0', 'request_id': 'req-162a5b98-a790-4116-b5b8-9f6a2d84b3e6'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:17:43.874245 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-162a5b98-a790-4116-b5b8-9f6a2d84b3e6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8f640560967147e3b567d705fc77fce0', 'request_id': 'req-162a5b98-a790-4116-b5b8-9f6a2d84b3e6'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:17:43.908572 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-162a5b98-a790-4116-b5b8-9f6a2d84b3e6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8f640560967147e3b567d705fc77fce0', 'request_id': 'req-162a5b98-a790-4116-b5b8-9f6a2d84b3e6'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:17:43.952232 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-162a5b98-a790-4116-b5b8-9f6a2d84b3e6 admin admin] The token cache is being invalidate because project 8f640560967147e3b567d705fc77fce0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80382) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 09:17:43.954425 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-162a5b98-a790-4116-b5b8-9f6a2d84b3e6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:17:43.960735 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-162a5b98-a790-4116-b5b8-9f6a2d84b3e6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:17:43.968026 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 833/1656] 10.222.0.26 () {58 vars in 1232 bytes} [Thu Jul 14 09:17:43 2022] DELETE /identity/v3/projects/8f640560967147e3b567d705fc77fce0 => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:17:43.974284 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811e7dd5-233d-44b2-ad2f-90d889c1584a None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:17:43.975301 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811e7dd5-233d-44b2-ad2f-90d889c1584a None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:17:43.975679 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811e7dd5-233d-44b2-ad2f-90d889c1584a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:17:44.037711 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-811e7dd5-233d-44b2-ad2f-90d889c1584a None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:17:44.042747 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-811e7dd5-233d-44b2-ad2f-90d889c1584a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:17:44.191916 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 824/1657] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:17:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:17:44.192999 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f982e40b-2d33-41c5-84ce-d54fb6e9387c None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:17:44.193316 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f982e40b-2d33-41c5-84ce-d54fb6e9387c None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:17:44.193656 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f982e40b-2d33-41c5-84ce-d54fb6e9387c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:17:44.232744 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-f982e40b-2d33-41c5-84ce-d54fb6e9387c None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:17:44.236251 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-f982e40b-2d33-41c5-84ce-d54fb6e9387c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:17:44.254719 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811e7dd5-233d-44b2-ad2f-90d889c1584a None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:17:44.267356 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 834/1658] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:17:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 09:17:44.279338 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811e7dd5-233d-44b2-ad2f-90d889c1584a None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:17:44.287920 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb90f6c-3f42-456c-81c0-e05cc3881f67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:17:44.307559 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb90f6c-3f42-456c-81c0-e05cc3881f67 admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:17:44.307976 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb90f6c-3f42-456c-81c0-e05cc3881f67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:17:44.308373 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb90f6c-3f42-456c-81c0-e05cc3881f67 admin admin] PATH_INFO: `/v3/domains` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:17:44.309349 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb90f6c-3f42-456c-81c0-e05cc3881f67 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:17:44.309751 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb90f6c-3f42-456c-81c0-e05cc3881f67 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:17:44.317769 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb90f6c-3f42-456c-81c0-e05cc3881f67 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:17:44.339436 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f982e40b-2d33-41c5-84ce-d54fb6e9387c None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:17:44.339542 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 825/1659] 10.222.0.26 () {58 vars in 1152 bytes} [Thu Jul 14 09:17:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:17:44.352977 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f982e40b-2d33-41c5-84ce-d54fb6e9387c None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:17:44.386074 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27958b74-aca2-49db-81b9-4115cc48f71b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:17:44.386947 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27958b74-aca2-49db-81b9-4115cc48f71b admin admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:17:44.387262 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27958b74-aca2-49db-81b9-4115cc48f71b admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:17:44.387536 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27958b74-aca2-49db-81b9-4115cc48f71b admin admin] PATH_INFO: `/v3/projects` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:17:44.389474 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27958b74-aca2-49db-81b9-4115cc48f71b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:17:44.398996 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27958b74-aca2-49db-81b9-4115cc48f71b admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:17:44.463289 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 835/1660] 10.222.0.26 () {60 vars in 1152 bytes} [Thu Jul 14 09:17:44 2022] POST /identity/v3/projects => generated 403 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 09:17:44.467240 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb90f6c-3f42-456c-81c0-e05cc3881f67 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:17:44.478569 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb90f6c-3f42-456c-81c0-e05cc3881f67 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:17:44.598631 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa82246-ead0-4274-8245-857fa5f83eed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:17:44.600164 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa82246-ead0-4274-8245-857fa5f83eed admin admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:17:44.600664 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa82246-ead0-4274-8245-857fa5f83eed admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:17:44.601164 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa82246-ead0-4274-8245-857fa5f83eed admin admin] PATH_INFO: `/v3/users` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:17:44.602537 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa82246-ead0-4274-8245-857fa5f83eed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:17:44.605902 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa82246-ead0-4274-8245-857fa5f83eed admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:17:44.643411 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27958b74-aca2-49db-81b9-4115cc48f71b admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:17:44.644539 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 826/1661] 10.222.0.26 () {60 vars in 1143 bytes} [Thu Jul 14 09:17:44 2022] POST /identity/v3/users => generated 302 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 09:17:44.660376 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27958b74-aca2-49db-81b9-4115cc48f71b admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:17:44.670445 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b438b7c9-7e13-40b7-87fc-88d1d2462a9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:17:44.671294 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b438b7c9-7e13-40b7-87fc-88d1d2462a9f admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:17:44.671654 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b438b7c9-7e13-40b7-87fc-88d1d2462a9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:17:44.672002 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b438b7c9-7e13-40b7-87fc-88d1d2462a9f admin admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:17:44.672877 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b438b7c9-7e13-40b7-87fc-88d1d2462a9f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:17:44.679166 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b438b7c9-7e13-40b7-87fc-88d1d2462a9f admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:17:44.704021 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 836/1662] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jul 14 09:17:44 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 09:17:44.708216 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa82246-ead0-4274-8245-857fa5f83eed admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:17:44.728151 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa82246-ead0-4274-8245-857fa5f83eed admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:17:44.743171 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc575136-6d6b-4ded-aab6-80163bbf8116 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:17:44.743713 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc575136-6d6b-4ded-aab6-80163bbf8116 admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:17:44.743998 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc575136-6d6b-4ded-aab6-80163bbf8116 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:17:44.744274 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc575136-6d6b-4ded-aab6-80163bbf8116 admin admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:17:44.745301 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc575136-6d6b-4ded-aab6-80163bbf8116 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:17:44.750785 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc575136-6d6b-4ded-aab6-80163bbf8116 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:17:44.756957 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:17:44.788552 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 827/1663] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jul 14 09:17:44 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 09:17:44.789323 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b438b7c9-7e13-40b7-87fc-88d1d2462a9f admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:17:44.810774 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b438b7c9-7e13-40b7-87fc-88d1d2462a9f admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:17:44.822351 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025b54c4-ccc7-4e41-89b4-80c34c297e03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:17:44.822498 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025b54c4-ccc7-4e41-89b4-80c34c297e03 admin admin] REQUEST_METHOD: `PUT` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:17:44.822692 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025b54c4-ccc7-4e41-89b4-80c34c297e03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:17:44.823133 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025b54c4-ccc7-4e41-89b4-80c34c297e03 admin admin] PATH_INFO: `/v3/projects/63da73ee220842e89a71f8436dffdd06/users/b212293150a44f5b89100607e2ed3dd2/roles/d1691322ce6a4edab7ce722edf0f2988` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:17:44.863991 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025b54c4-ccc7-4e41-89b4-80c34c297e03 admin admin] RBAC: Authorizing `identity:create_grant(project_id=63da73ee220842e89a71f8436dffdd06, user_id=b212293150a44f5b89100607e2ed3dd2, role_id=d1691322ce6a4edab7ce722edf0f2988)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:17:44.870392 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025b54c4-ccc7-4e41-89b4-80c34c297e03 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:17:44.941523 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 837/1664] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jul 14 09:17:44 2022] PUT /identity/v3/projects/63da73ee220842e89a71f8436dffdd06/users/b212293150a44f5b89100607e2ed3dd2/roles/d1691322ce6a4edab7ce722edf0f2988 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:17:44.947103 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74990e4-7809-4298-b470-5a2f241bb00c None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:17:44.947103 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74990e4-7809-4298-b470-5a2f241bb00c None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:17:44.947103 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74990e4-7809-4298-b470-5a2f241bb00c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:17:45.004941 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-a74990e4-7809-4298-b470-5a2f241bb00c None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:17:45.009944 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-a74990e4-7809-4298-b470-5a2f241bb00c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:17:45.191650 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 828/1665] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:17:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:17:45.211011 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025b54c4-ccc7-4e41-89b4-80c34c297e03 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:17:45.212895 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-025b54c4-ccc7-4e41-89b4-80c34c297e03 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:17:45.363146 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025b54c4-ccc7-4e41-89b4-80c34c297e03 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:17:45.367677 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec2a292-bac2-4d6e-b60f-78b41a9a0725 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '952911c492444ffea4b211a24b564117', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:17:45.368514 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec2a292-bac2-4d6e-b60f-78b41a9a0725 service neutron] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:17:45.368919 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec2a292-bac2-4d6e-b60f-78b41a9a0725 service neutron] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:17:45.369295 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec2a292-bac2-4d6e-b60f-78b41a9a0725 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:17:45.406290 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec2a292-bac2-4d6e-b60f-78b41a9a0725 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:17:45.408220 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec2a292-bac2-4d6e-b60f-78b41a9a0725 service neutron] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:17:45.434242 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 838/1666] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:17:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:17:46.354100 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74990e4-7809-4298-b470-5a2f241bb00c None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:17:46.357273 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-a74990e4-7809-4298-b470-5a2f241bb00c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:17:46.392900 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74990e4-7809-4298-b470-5a2f241bb00c None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:17:46.398178 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbcabd4-ee8b-42d9-ab14-b11a50d034ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:17:46.399092 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbcabd4-ee8b-42d9-ab14-b11a50d034ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:17:46.399950 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbcabd4-ee8b-42d9-ab14-b11a50d034ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:17:46.400237 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbcabd4-ee8b-42d9-ab14-b11a50d034ed admin admin] PATH_INFO: `/v3/users/41a1046e199343b0a1a727ce6025260a` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:17:46.401882 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbcabd4-ee8b-42d9-ab14-b11a50d034ed admin admin] RBAC: Authorizing `identity:delete_user(user_id=41a1046e199343b0a1a727ce6025260a)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:17:46.409289 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbcabd4-ee8b-42d9-ab14-b11a50d034ed admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:17:46.495693 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-3fbcabd4-ee8b-42d9-ab14-b11a50d034ed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '41a1046e199343b0a1a727ce6025260a'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:17:46.508836 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-3fbcabd4-ee8b-42d9-ab14-b11a50d034ed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '41a1046e199343b0a1a727ce6025260a'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:17:46.509810 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-3fbcabd4-ee8b-42d9-ab14-b11a50d034ed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '41a1046e199343b0a1a727ce6025260a'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:17:46.510864 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-3fbcabd4-ee8b-42d9-ab14-b11a50d034ed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '41a1046e199343b0a1a727ce6025260a'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:17:46.538239 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-3fbcabd4-ee8b-42d9-ab14-b11a50d034ed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '41a1046e199343b0a1a727ce6025260a'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:17:46.580126 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 829/1667] 10.222.0.26 () {58 vars in 1223 bytes} [Thu Jul 14 09:17:46 2022] DELETE /identity/v3/users/41a1046e199343b0a1a727ce6025260a => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:17:46.585349 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf4468c-b7f7-4160-9000-363efc5cd19f None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:17:46.585973 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf4468c-b7f7-4160-9000-363efc5cd19f None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:17:46.585973 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf4468c-b7f7-4160-9000-363efc5cd19f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:17:46.616560 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-bcf4468c-b7f7-4160-9000-363efc5cd19f None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:17:46.620495 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-bcf4468c-b7f7-4160-9000-363efc5cd19f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:17:46.697930 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:17:46.823287 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 839/1668] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:17:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 09:17:46.848780 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbcabd4-ee8b-42d9-ab14-b11a50d034ed admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:17:46.850296 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-3fbcabd4-ee8b-42d9-ab14-b11a50d034ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:17:46.896505 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf4468c-b7f7-4160-9000-363efc5cd19f None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:17:46.898215 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-bcf4468c-b7f7-4160-9000-363efc5cd19f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:17:47.065282 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf4468c-b7f7-4160-9000-363efc5cd19f None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:17:47.072297 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95d617f-8810-4693-a024-d48522c208e6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e76f1952a5b146808844b9f2b7395035', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:17:47.073004 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95d617f-8810-4693-a024-d48522c208e6 service cinder] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:17:47.073622 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95d617f-8810-4693-a024-d48522c208e6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:17:47.073622 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95d617f-8810-4693-a024-d48522c208e6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:17:47.082446 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-e95d617f-8810-4693-a024-d48522c208e6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:17:47.096925 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbcabd4-ee8b-42d9-ab14-b11a50d034ed admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:17:47.108178 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625faeca-cfb4-451e-b2f6-6aca48355d1a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '952911c492444ffea4b211a24b564117', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:17:47.116752 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625faeca-cfb4-451e-b2f6-6aca48355d1a service neutron] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:17:47.119035 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625faeca-cfb4-451e-b2f6-6aca48355d1a service neutron] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:17:47.119462 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625faeca-cfb4-451e-b2f6-6aca48355d1a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:17:47.159430 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625faeca-cfb4-451e-b2f6-6aca48355d1a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:17:47.161752 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625faeca-cfb4-451e-b2f6-6aca48355d1a service neutron] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:17:47.191850 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 830/1669] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:17:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:17:47.245861 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95d617f-8810-4693-a024-d48522c208e6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:17:47.247874 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95d617f-8810-4693-a024-d48522c208e6 service cinder] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:17:47.278268 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 840/1670] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:17:46 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 09:17:48.228352 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b8a53a-4e89-4868-8325-725bb3994d0c None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:17:48.229895 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b8a53a-4e89-4868-8325-725bb3994d0c None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:17:48.230340 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b8a53a-4e89-4868-8325-725bb3994d0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:17:48.277198 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:17:48.285110 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-e8b8a53a-4e89-4868-8325-725bb3994d0c None None] MFA Rules not processed for user `4d619bfc0939493480fa98451db2f30f`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:17:48.287779 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-e8b8a53a-4e89-4868-8325-725bb3994d0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:17:48.444969 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 831/1671] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:17:48 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:17:48.465717 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95d617f-8810-4693-a024-d48522c208e6 service cinder] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:17:48.467322 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-e95d617f-8810-4693-a024-d48522c208e6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:17:48.602161 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95d617f-8810-4693-a024-d48522c208e6 service cinder] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:17:48.608903 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e8b820-6a3d-40d0-b126-7896570147f3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '538fff3c5a2b487eb44ae7c1a1198102', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:17:48.610501 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e8b820-6a3d-40d0-b126-7896570147f3 service nova] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:17:48.610951 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e8b820-6a3d-40d0-b126-7896570147f3 service nova] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:17:48.611407 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e8b820-6a3d-40d0-b126-7896570147f3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:17:48.625280 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e8b820-6a3d-40d0-b126-7896570147f3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:17:48.627172 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e8b820-6a3d-40d0-b126-7896570147f3 service nova] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:17:48.641121 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 841/1672] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:17:48 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:17:48.766307 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da240652-6d56-4c82-a35f-70029629047c None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:17:48.766651 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da240652-6d56-4c82-a35f-70029629047c None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:17:48.767735 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da240652-6d56-4c82-a35f-70029629047c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:17:48.830268 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-da240652-6d56-4c82-a35f-70029629047c None None] MFA Rules not processed for user `b212293150a44f5b89100607e2ed3dd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:17:48.831643 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-da240652-6d56-4c82-a35f-70029629047c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:17:48.971136 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 832/1673] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:17:48 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:17:48.976545 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e8b820-6a3d-40d0-b126-7896570147f3 service nova] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:17:48.977746 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-91e8b820-6a3d-40d0-b126-7896570147f3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:17:48.989624 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:17:49.002412 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e8b820-6a3d-40d0-b126-7896570147f3 service nova] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:17:49.012532 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707ebc71-4d8e-45f5-945c-1e29ed84c151 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:17:49.014384 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707ebc71-4d8e-45f5-945c-1e29ed84c151 admin admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:17:49.014697 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707ebc71-4d8e-45f5-945c-1e29ed84c151 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:17:49.014951 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707ebc71-4d8e-45f5-945c-1e29ed84c151 admin admin] PATH_INFO: `/v3/projects` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:17:49.015702 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707ebc71-4d8e-45f5-945c-1e29ed84c151 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:17:49.021350 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707ebc71-4d8e-45f5-945c-1e29ed84c151 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:17:49.086647 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 842/1674] 10.222.0.26 () {60 vars in 1152 bytes} [Thu Jul 14 09:17:48 2022] POST /identity/v3/projects => generated 401 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 09:17:49.087847 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da240652-6d56-4c82-a35f-70029629047c None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:17:49.115517 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da240652-6d56-4c82-a35f-70029629047c None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:17:49.292369 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b17070-d636-4f35-a6ae-fdec93617429 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:17:49.297915 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b17070-d636-4f35-a6ae-fdec93617429 admin admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:17:49.298379 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b17070-d636-4f35-a6ae-fdec93617429 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:17:49.298797 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b17070-d636-4f35-a6ae-fdec93617429 admin admin] PATH_INFO: `/v3/users` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:17:49.300151 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b17070-d636-4f35-a6ae-fdec93617429 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:17:49.302274 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b17070-d636-4f35-a6ae-fdec93617429 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:17:49.355855 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 833/1675] 10.222.0.26 () {60 vars in 1143 bytes} [Thu Jul 14 09:17:49 2022] POST /identity/v3/users => generated 307 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 09:17:49.362833 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707ebc71-4d8e-45f5-945c-1e29ed84c151 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:17:49.382053 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707ebc71-4d8e-45f5-945c-1e29ed84c151 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:17:49.401923 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3b852e-79a5-42bc-91fd-f039202e71c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:17:49.402713 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3b852e-79a5-42bc-91fd-f039202e71c5 admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:17:49.404211 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3b852e-79a5-42bc-91fd-f039202e71c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:17:49.404500 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3b852e-79a5-42bc-91fd-f039202e71c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:17:49.406069 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3b852e-79a5-42bc-91fd-f039202e71c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:17:49.411136 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3b852e-79a5-42bc-91fd-f039202e71c5 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:17:49.434682 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 843/1676] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jul 14 09:17:49 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 09:17:49.439167 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b17070-d636-4f35-a6ae-fdec93617429 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:17:49.457732 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3b852e-79a5-42bc-91fd-f039202e71c5 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:17:49.459641 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-2e3b852e-79a5-42bc-91fd-f039202e71c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:17:49.473125 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b17070-d636-4f35-a6ae-fdec93617429 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:17:49.502939 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd8ff54-bb52-464a-b136-77da4f275018 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:17:49.505315 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3b852e-79a5-42bc-91fd-f039202e71c5 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:17:49.507926 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd8ff54-bb52-464a-b136-77da4f275018 admin admin] REQUEST_METHOD: `PUT` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:17:49.508212 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd8ff54-bb52-464a-b136-77da4f275018 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:17:49.511286 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd8ff54-bb52-464a-b136-77da4f275018 admin admin] PATH_INFO: `/v3/projects/d294bb86d04e4d5386e441ed183fa9ae/users/929b981098ba4ee18b306e21bfb18424/roles/7b549d58d1724395b343609d895c80d0` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:17:49.526237 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd3e999-bbae-4010-8988-b80e6064cc64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:17:49.537100 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd3e999-bbae-4010-8988-b80e6064cc64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:17:49.537528 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd3e999-bbae-4010-8988-b80e6064cc64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:17:49.537819 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd3e999-bbae-4010-8988-b80e6064cc64 admin admin] PATH_INFO: `/v3/projects/5fef7aa711754cff8e9e1b50665df7ae` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:17:49.547962 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd3e999-bbae-4010-8988-b80e6064cc64 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5fef7aa711754cff8e9e1b50665df7ae)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:17:49.563729 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd3e999-bbae-4010-8988-b80e6064cc64 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:17:49.581723 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd8ff54-bb52-464a-b136-77da4f275018 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d294bb86d04e4d5386e441ed183fa9ae, user_id=929b981098ba4ee18b306e21bfb18424, role_id=7b549d58d1724395b343609d895c80d0)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:17:49.596496 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd8ff54-bb52-464a-b136-77da4f275018 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:17:49.659744 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:17:49.676679 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 834/1677] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jul 14 09:17:49 2022] PUT /identity/v3/projects/d294bb86d04e4d5386e441ed183fa9ae/users/929b981098ba4ee18b306e21bfb18424/roles/7b549d58d1724395b343609d895c80d0 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:17:49.678069 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd8ff54-bb52-464a-b136-77da4f275018 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:17:49.753484 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-6fd3e999-bbae-4010-8988-b80e6064cc64 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5fef7aa711754cff8e9e1b50665df7ae', 'request_id': 'req-6fd3e999-bbae-4010-8988-b80e6064cc64'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:17:49.767888 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-6fd3e999-bbae-4010-8988-b80e6064cc64 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5fef7aa711754cff8e9e1b50665df7ae', 'request_id': 'req-6fd3e999-bbae-4010-8988-b80e6064cc64'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:17:49.783867 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-6fd3e999-bbae-4010-8988-b80e6064cc64 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5fef7aa711754cff8e9e1b50665df7ae', 'request_id': 'req-6fd3e999-bbae-4010-8988-b80e6064cc64'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:17:49.807235 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-6fd3e999-bbae-4010-8988-b80e6064cc64 admin admin] The token cache is being invalidate because project 5fef7aa711754cff8e9e1b50665df7ae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80382) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 09:17:49.808266 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-6fd3e999-bbae-4010-8988-b80e6064cc64 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:17:49.809540 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-6fd3e999-bbae-4010-8988-b80e6064cc64 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:17:49.813531 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 844/1678] 10.222.0.26 () {58 vars in 1232 bytes} [Thu Jul 14 09:17:49 2022] DELETE /identity/v3/projects/5fef7aa711754cff8e9e1b50665df7ae => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:17:49.817483 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd3e999-bbae-4010-8988-b80e6064cc64 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:17:49.819687 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-6fd3e999-bbae-4010-8988-b80e6064cc64 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:17:49.846942 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd8ff54-bb52-464a-b136-77da4f275018 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:17:49.986912 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b1b4b1-366e-4caf-8b18-8f35de12cc49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:17:49.987774 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b1b4b1-366e-4caf-8b18-8f35de12cc49 admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:17:49.988067 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b1b4b1-366e-4caf-8b18-8f35de12cc49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:17:49.988346 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b1b4b1-366e-4caf-8b18-8f35de12cc49 admin admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:17:49.989101 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b1b4b1-366e-4caf-8b18-8f35de12cc49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:17:49.990737 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b1b4b1-366e-4caf-8b18-8f35de12cc49 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:17:49.992267 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd3e999-bbae-4010-8988-b80e6064cc64 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:17:50.009275 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2ec758-7a71-4bb2-a4ad-3d6ba9a8bf70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:17:50.012654 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2ec758-7a71-4bb2-a4ad-3d6ba9a8bf70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:17:50.013019 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2ec758-7a71-4bb2-a4ad-3d6ba9a8bf70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:17:50.014111 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2ec758-7a71-4bb2-a4ad-3d6ba9a8bf70 admin admin] PATH_INFO: `/v3/users/96ea3b17dbea446bb961dcea712ad169` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:17:50.016371 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2ec758-7a71-4bb2-a4ad-3d6ba9a8bf70 admin admin] RBAC: Authorizing `identity:delete_user(user_id=96ea3b17dbea446bb961dcea712ad169)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:17:50.017355 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 835/1679] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jul 14 09:17:49 2022] GET /identity/v3/roles => generated 1420 bytes in 340 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 09:17:50.019760 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: WARNING py.warnings [None req-5d2ec758-7a71-4bb2-a4ad-3d6ba9a8bf70 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 14 09:17:50.019760 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: warnings.warn(msg) Jul 14 09:17:50.019760 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: Jul 14 09:17:50.023606 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2ec758-7a71-4bb2-a4ad-3d6ba9a8bf70 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:17:50.024206 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b1b4b1-366e-4caf-8b18-8f35de12cc49 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:17:50.028422 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-13b1b4b1-366e-4caf-8b18-8f35de12cc49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:17:50.057692 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b1b4b1-366e-4caf-8b18-8f35de12cc49 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:17:50.071911 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da2aac3-c4b6-47a6-9c93-02783c64cbdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:17:50.072978 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da2aac3-c4b6-47a6-9c93-02783c64cbdd admin admin] REQUEST_METHOD: `PUT` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:17:50.073346 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da2aac3-c4b6-47a6-9c93-02783c64cbdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:17:50.073627 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da2aac3-c4b6-47a6-9c93-02783c64cbdd admin admin] PATH_INFO: `/v3/projects/d294bb86d04e4d5386e441ed183fa9ae/users/929b981098ba4ee18b306e21bfb18424/roles/7b549d58d1724395b343609d895c80d0` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:17:50.077776 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da2aac3-c4b6-47a6-9c93-02783c64cbdd admin admin] RBAC: Authorizing `identity:create_grant(project_id=d294bb86d04e4d5386e441ed183fa9ae, user_id=929b981098ba4ee18b306e21bfb18424, role_id=7b549d58d1724395b343609d895c80d0)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:17:50.084509 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da2aac3-c4b6-47a6-9c93-02783c64cbdd admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:17:50.116124 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 836/1680] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jul 14 09:17:50 2022] PUT /identity/v3/projects/d294bb86d04e4d5386e441ed183fa9ae/users/929b981098ba4ee18b306e21bfb18424/roles/7b549d58d1724395b343609d895c80d0 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:17:50.190358 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-5d2ec758-7a71-4bb2-a4ad-3d6ba9a8bf70 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '96ea3b17dbea446bb961dcea712ad169'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:17:50.191581 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-5d2ec758-7a71-4bb2-a4ad-3d6ba9a8bf70 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '96ea3b17dbea446bb961dcea712ad169'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:17:50.214745 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=83,keystone:INSERT=2,keystone:DELETE=15 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:17:50.240410 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-5d2ec758-7a71-4bb2-a4ad-3d6ba9a8bf70 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '96ea3b17dbea446bb961dcea712ad169'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:17:50.258606 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-5d2ec758-7a71-4bb2-a4ad-3d6ba9a8bf70 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '96ea3b17dbea446bb961dcea712ad169'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:17:50.259573 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-5d2ec758-7a71-4bb2-a4ad-3d6ba9a8bf70 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '96ea3b17dbea446bb961dcea712ad169'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:17:50.327246 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 845/1681] 10.222.0.26 () {58 vars in 1223 bytes} [Thu Jul 14 09:17:49 2022] DELETE /identity/v3/users/96ea3b17dbea446bb961dcea712ad169 => generated 0 bytes in 510 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:17:51.160349 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a798f0b2-1206-4089-a871-f7b7355b1002 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:17:51.160886 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a798f0b2-1206-4089-a871-f7b7355b1002 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:17:51.161120 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a798f0b2-1206-4089-a871-f7b7355b1002 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:17:51.218837 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-a798f0b2-1206-4089-a871-f7b7355b1002 None None] MFA Rules not processed for user `18377cf2b2d14cb0b4cbda6ec720ed28`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:17:51.220100 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-a798f0b2-1206-4089-a871-f7b7355b1002 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:17:51.411266 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 837/1682] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:17:51 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:17:51.412679 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07bcb75-6198-4595-9439-fbda7db5e55c None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:17:51.413169 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07bcb75-6198-4595-9439-fbda7db5e55c None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:17:51.413294 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07bcb75-6198-4595-9439-fbda7db5e55c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:17:51.446294 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-d07bcb75-6198-4595-9439-fbda7db5e55c None None] MFA Rules not processed for user `18377cf2b2d14cb0b4cbda6ec720ed28`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:17:51.451933 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-d07bcb75-6198-4595-9439-fbda7db5e55c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:17:51.475218 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 846/1683] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:17:51 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:17:51.487389 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a798f0b2-1206-4089-a871-f7b7355b1002 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:17:51.489210 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-a798f0b2-1206-4089-a871-f7b7355b1002 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:17:51.538882 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:17:51.678053 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a798f0b2-1206-4089-a871-f7b7355b1002 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:17:51.706827 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92141c7c-7ab5-48f4-a97e-63780f8a462a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '538fff3c5a2b487eb44ae7c1a1198102', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:17:51.707571 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92141c7c-7ab5-48f4-a97e-63780f8a462a service nova] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:17:51.707881 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92141c7c-7ab5-48f4-a97e-63780f8a462a service nova] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:17:51.708127 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92141c7c-7ab5-48f4-a97e-63780f8a462a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:17:51.744050 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92141c7c-7ab5-48f4-a97e-63780f8a462a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:17:51.745781 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92141c7c-7ab5-48f4-a97e-63780f8a462a service nova] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:17:51.786241 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 838/1684] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:17:51 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:17:51.787099 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07bcb75-6198-4595-9439-fbda7db5e55c None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:17:51.788846 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-d07bcb75-6198-4595-9439-fbda7db5e55c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:17:51.974116 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07bcb75-6198-4595-9439-fbda7db5e55c None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:17:51.988753 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a582aa02-2891-494d-9a72-86a4eb5cd13d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:17:51.989689 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a582aa02-2891-494d-9a72-86a4eb5cd13d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:17:51.995360 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a582aa02-2891-494d-9a72-86a4eb5cd13d admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:17:51.995465 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a582aa02-2891-494d-9a72-86a4eb5cd13d admin admin] PATH_INFO: `/v3/projects/c779db3b2b21484abc448a330bd36c79` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:17:52.001692 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a582aa02-2891-494d-9a72-86a4eb5cd13d admin admin] RBAC: Authorizing `identity:delete_project(project_id=c779db3b2b21484abc448a330bd36c79)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:17:52.008252 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a582aa02-2891-494d-9a72-86a4eb5cd13d admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:17:52.168162 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-a582aa02-2891-494d-9a72-86a4eb5cd13d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c779db3b2b21484abc448a330bd36c79', 'request_id': 'req-a582aa02-2891-494d-9a72-86a4eb5cd13d'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:17:52.178095 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-a582aa02-2891-494d-9a72-86a4eb5cd13d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c779db3b2b21484abc448a330bd36c79', 'request_id': 'req-a582aa02-2891-494d-9a72-86a4eb5cd13d'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:17:52.187567 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-a582aa02-2891-494d-9a72-86a4eb5cd13d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c779db3b2b21484abc448a330bd36c79', 'request_id': 'req-a582aa02-2891-494d-9a72-86a4eb5cd13d'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:17:52.213675 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-a582aa02-2891-494d-9a72-86a4eb5cd13d admin admin] The token cache is being invalidate because project c779db3b2b21484abc448a330bd36c79 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80382) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 09:17:52.214396 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-a582aa02-2891-494d-9a72-86a4eb5cd13d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:17:52.215816 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-a582aa02-2891-494d-9a72-86a4eb5cd13d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:17:52.218902 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 847/1685] 10.222.0.26 () {58 vars in 1232 bytes} [Thu Jul 14 09:17:51 2022] DELETE /identity/v3/projects/c779db3b2b21484abc448a330bd36c79 => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:17:52.224057 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf68b0cf-e0b5-4d01-8e6c-9abc9b071492 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:17:52.224546 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf68b0cf-e0b5-4d01-8e6c-9abc9b071492 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:17:52.224630 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf68b0cf-e0b5-4d01-8e6c-9abc9b071492 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:17:52.265935 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-bf68b0cf-e0b5-4d01-8e6c-9abc9b071492 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:17:52.269338 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-bf68b0cf-e0b5-4d01-8e6c-9abc9b071492 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:17:52.424645 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 839/1686] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:17:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 09:17:52.432694 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0cef34-9932-4e15-a373-ba7fb0b3fe02 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:17:52.433281 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0cef34-9932-4e15-a373-ba7fb0b3fe02 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:17:52.433281 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0cef34-9932-4e15-a373-ba7fb0b3fe02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:17:52.460793 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-4d0cef34-9932-4e15-a373-ba7fb0b3fe02 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:17:52.462420 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-4d0cef34-9932-4e15-a373-ba7fb0b3fe02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:17:52.481131 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf68b0cf-e0b5-4d01-8e6c-9abc9b071492 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:17:52.488260 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 848/1687] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:17:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:17:52.494233 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf68b0cf-e0b5-4d01-8e6c-9abc9b071492 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:17:52.500703 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ec56e1-067a-4373-8587-1c1788594fd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:17:52.501645 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ec56e1-067a-4373-8587-1c1788594fd0 admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:17:52.502194 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ec56e1-067a-4373-8587-1c1788594fd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:17:52.502596 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ec56e1-067a-4373-8587-1c1788594fd0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:17:52.503658 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ec56e1-067a-4373-8587-1c1788594fd0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:17:52.504097 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ec56e1-067a-4373-8587-1c1788594fd0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:17:52.506104 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ec56e1-067a-4373-8587-1c1788594fd0 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:17:52.529771 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0cef34-9932-4e15-a373-ba7fb0b3fe02 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:17:52.537240 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 840/1688] 10.222.0.26 () {58 vars in 1152 bytes} [Thu Jul 14 09:17:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:17:52.542496 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0cef34-9932-4e15-a373-ba7fb0b3fe02 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:17:52.551735 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47857b4c-ce30-4a98-8b5f-94b9e688443a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:17:52.553243 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47857b4c-ce30-4a98-8b5f-94b9e688443a admin admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:17:52.553991 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47857b4c-ce30-4a98-8b5f-94b9e688443a admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:17:52.553991 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47857b4c-ce30-4a98-8b5f-94b9e688443a admin admin] PATH_INFO: `/v3/projects` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:17:52.555295 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47857b4c-ce30-4a98-8b5f-94b9e688443a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:17:52.557568 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47857b4c-ce30-4a98-8b5f-94b9e688443a admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:17:52.608046 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ec56e1-067a-4373-8587-1c1788594fd0 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:17:52.616170 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 849/1689] 10.222.0.26 () {60 vars in 1152 bytes} [Thu Jul 14 09:17:52 2022] POST /identity/v3/projects => generated 367 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 09:17:52.622274 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ec56e1-067a-4373-8587-1c1788594fd0 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:17:52.758906 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e923c996-9544-467e-83f8-bfb680ede933 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:17:52.759767 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e923c996-9544-467e-83f8-bfb680ede933 admin admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:17:52.760150 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e923c996-9544-467e-83f8-bfb680ede933 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:17:52.761736 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e923c996-9544-467e-83f8-bfb680ede933 admin admin] PATH_INFO: `/v3/users` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:17:52.763716 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e923c996-9544-467e-83f8-bfb680ede933 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:17:52.766484 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e923c996-9544-467e-83f8-bfb680ede933 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:17:52.814858 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 841/1690] 10.222.0.26 () {60 vars in 1143 bytes} [Thu Jul 14 09:17:52 2022] POST /identity/v3/users => generated 284 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 09:17:52.820471 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47857b4c-ce30-4a98-8b5f-94b9e688443a admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:17:52.845690 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47857b4c-ce30-4a98-8b5f-94b9e688443a admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:17:52.856212 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7865980-4fbc-4d8c-a563-3eed93d0653d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:17:52.860123 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7865980-4fbc-4d8c-a563-3eed93d0653d admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:17:52.861555 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7865980-4fbc-4d8c-a563-3eed93d0653d admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:17:52.861866 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7865980-4fbc-4d8c-a563-3eed93d0653d admin admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:17:52.864531 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7865980-4fbc-4d8c-a563-3eed93d0653d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:17:52.869290 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7865980-4fbc-4d8c-a563-3eed93d0653d admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:17:52.910137 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 850/1691] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jul 14 09:17:52 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 09:17:52.910904 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e923c996-9544-467e-83f8-bfb680ede933 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:17:52.930368 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e923c996-9544-467e-83f8-bfb680ede933 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:17:52.936202 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b62a037-fe10-40b1-9130-87ac2c0054b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:17:52.936895 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b62a037-fe10-40b1-9130-87ac2c0054b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:17:52.937179 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b62a037-fe10-40b1-9130-87ac2c0054b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:17:52.937480 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b62a037-fe10-40b1-9130-87ac2c0054b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:17:52.938195 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b62a037-fe10-40b1-9130-87ac2c0054b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:17:52.939972 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b62a037-fe10-40b1-9130-87ac2c0054b1 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:17:52.980889 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 842/1692] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jul 14 09:17:52 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 09:17:52.983737 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7865980-4fbc-4d8c-a563-3eed93d0653d admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:17:53.016912 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7865980-4fbc-4d8c-a563-3eed93d0653d admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:17:53.021382 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be273324-332a-4f7f-ae52-7668fc5632d9 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:17:53.021724 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be273324-332a-4f7f-ae52-7668fc5632d9 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:17:53.022015 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be273324-332a-4f7f-ae52-7668fc5632d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:17:53.037969 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1219a966-4977-4dce-bbfc-02c7e4231c9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:17:53.042652 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1219a966-4977-4dce-bbfc-02c7e4231c9e admin admin] REQUEST_METHOD: `PUT` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:17:53.042652 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1219a966-4977-4dce-bbfc-02c7e4231c9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:17:53.042652 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1219a966-4977-4dce-bbfc-02c7e4231c9e admin admin] PATH_INFO: `/v3/projects/db823124e7704309be3370a64fcde324/users/bf084512b5bd4444adfd200e44064741/roles/d1691322ce6a4edab7ce722edf0f2988` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:17:53.057370 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:17:53.077718 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-be273324-332a-4f7f-ae52-7668fc5632d9 None None] MFA Rules not processed for user `854fc6f464c940a78f779438aa3ee8a9`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:17:53.079099 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-be273324-332a-4f7f-ae52-7668fc5632d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:17:53.085354 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1219a966-4977-4dce-bbfc-02c7e4231c9e admin admin] RBAC: Authorizing `identity:create_grant(project_id=db823124e7704309be3370a64fcde324, user_id=bf084512b5bd4444adfd200e44064741, role_id=d1691322ce6a4edab7ce722edf0f2988)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:17:53.088093 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1219a966-4977-4dce-bbfc-02c7e4231c9e admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:17:53.090853 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=88,keystone:DELETE=9,keystone:INSERT=3 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:17:53.128740 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 851/1693] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jul 14 09:17:52 2022] PUT /identity/v3/projects/db823124e7704309be3370a64fcde324/users/bf084512b5bd4444adfd200e44064741/roles/d1691322ce6a4edab7ce722edf0f2988 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:17:53.134821 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa740b78-2c3c-426e-b0c3-cd7fc3ad5623 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:17:53.134983 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa740b78-2c3c-426e-b0c3-cd7fc3ad5623 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:17:53.135311 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa740b78-2c3c-426e-b0c3-cd7fc3ad5623 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:17:53.181550 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-aa740b78-2c3c-426e-b0c3-cd7fc3ad5623 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:17:53.182847 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-aa740b78-2c3c-426e-b0c3-cd7fc3ad5623 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:17:53.262843 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 843/1694] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:17:53 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 09:17:53.273918 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cc5b79-78ca-4fdf-999c-4354bf64ba7d None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:17:53.273918 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cc5b79-78ca-4fdf-999c-4354bf64ba7d None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:17:53.274365 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cc5b79-78ca-4fdf-999c-4354bf64ba7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:17:53.326382 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-41cc5b79-78ca-4fdf-999c-4354bf64ba7d None None] MFA Rules not processed for user `854fc6f464c940a78f779438aa3ee8a9`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:17:53.328374 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-41cc5b79-78ca-4fdf-999c-4354bf64ba7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:17:53.410554 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 852/1695] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:17:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 09:17:53.429762 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa740b78-2c3c-426e-b0c3-cd7fc3ad5623 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:17:53.432068 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-aa740b78-2c3c-426e-b0c3-cd7fc3ad5623 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:17:53.461594 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 844/1696] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:17:53 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 09:17:53.477048 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cc5b79-78ca-4fdf-999c-4354bf64ba7d None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:17:53.478771 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-41cc5b79-78ca-4fdf-999c-4354bf64ba7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:17:53.597058 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa740b78-2c3c-426e-b0c3-cd7fc3ad5623 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:17:53.609741 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50aa2502-c641-4e18-bf05-a217c9662dc9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '952911c492444ffea4b211a24b564117', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:17:53.610481 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50aa2502-c641-4e18-bf05-a217c9662dc9 service neutron] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:17:53.610765 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50aa2502-c641-4e18-bf05-a217c9662dc9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:17:53.611038 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50aa2502-c641-4e18-bf05-a217c9662dc9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:17:53.653397 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50aa2502-c641-4e18-bf05-a217c9662dc9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:17:53.654905 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50aa2502-c641-4e18-bf05-a217c9662dc9 service neutron] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:17:53.687385 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 853/1697] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:17:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 259 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 09:17:53.696195 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69b9b3c-0cbb-41c3-a243-50b97da35e31 None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:17:53.696488 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69b9b3c-0cbb-41c3-a243-50b97da35e31 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:17:53.696770 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69b9b3c-0cbb-41c3-a243-50b97da35e31 None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:17:53.698698 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cc5b79-78ca-4fdf-999c-4354bf64ba7d None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:17:53.699275 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 854/1698] 10.222.0.26 () {54 vars in 912 bytes} [Thu Jul 14 09:17:53 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:17:53.704738 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddd4af4-6fd4-42c3-aa3b-06d830d7cdd9 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:17:53.705027 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddd4af4-6fd4-42c3-aa3b-06d830d7cdd9 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:17:53.705306 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddd4af4-6fd4-42c3-aa3b-06d830d7cdd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:17:53.705703 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6775cf9-7864-4fac-9f87-bf6ee87a0b15 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '538fff3c5a2b487eb44ae7c1a1198102', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:17:53.706366 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6775cf9-7864-4fac-9f87-bf6ee87a0b15 service nova] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:17:53.706647 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6775cf9-7864-4fac-9f87-bf6ee87a0b15 service nova] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:17:53.706919 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6775cf9-7864-4fac-9f87-bf6ee87a0b15 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:17:53.716554 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6775cf9-7864-4fac-9f87-bf6ee87a0b15 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:17:53.718489 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6775cf9-7864-4fac-9f87-bf6ee87a0b15 service nova] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:17:53.733942 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 845/1699] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:17:53 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 263 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:17:53.799341 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-6ddd4af4-6fd4-42c3-aa3b-06d830d7cdd9 None None] MFA Rules not processed for user `952911c492444ffea4b211a24b564117`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:17:53.800586 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-6ddd4af4-6fd4-42c3-aa3b-06d830d7cdd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:17:53.829210 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 855/1700] 10.222.0.26 () {58 vars in 1009 bytes} [Thu Jul 14 09:17:53 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 09:17:53.870180 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6775cf9-7864-4fac-9f87-bf6ee87a0b15 service nova] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:17:53.872889 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-f6775cf9-7864-4fac-9f87-bf6ee87a0b15 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:17:53.902396 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6775cf9-7864-4fac-9f87-bf6ee87a0b15 service nova] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:17:53.944146 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b17f8e-a08c-4bf5-baa3-d06a80879f30 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '952911c492444ffea4b211a24b564117', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:17:53.952290 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b17f8e-a08c-4bf5-baa3-d06a80879f30 service neutron] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:17:53.953058 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b17f8e-a08c-4bf5-baa3-d06a80879f30 service neutron] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:17:53.953518 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b17f8e-a08c-4bf5-baa3-d06a80879f30 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:17:53.970100 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b17f8e-a08c-4bf5-baa3-d06a80879f30 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:17:53.974692 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b17f8e-a08c-4bf5-baa3-d06a80879f30 service neutron] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:17:53.997961 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 846/1701] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:17:53 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:17:54.219104 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef062744-391a-4a19-8351-c1bad73432e8 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:17:54.219431 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef062744-391a-4a19-8351-c1bad73432e8 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:17:54.219711 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef062744-391a-4a19-8351-c1bad73432e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:17:54.275548 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-ef062744-391a-4a19-8351-c1bad73432e8 None None] MFA Rules not processed for user `a2ab766e96024b899a01c540d9bd1322`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:17:54.276117 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d25e24b-5d97-4369-b51f-854f57a091a2 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:17:54.276851 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-ef062744-391a-4a19-8351-c1bad73432e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:17:54.277296 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d25e24b-5d97-4369-b51f-854f57a091a2 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:17:54.278073 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d25e24b-5d97-4369-b51f-854f57a091a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:17:54.349372 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-4d25e24b-5d97-4369-b51f-854f57a091a2 None None] MFA Rules not processed for user `4d619bfc0939493480fa98451db2f30f`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:17:54.350803 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-4d25e24b-5d97-4369-b51f-854f57a091a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:17:54.364255 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:17:54.481506 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 856/1702] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:17:54 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:17:54.509509 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 847/1703] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:17:54 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:17:54.674289 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a36e77-009e-4e20-907c-33044936c780 None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:17:54.674821 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a36e77-009e-4e20-907c-33044936c780 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:17:54.675369 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a36e77-009e-4e20-907c-33044936c780 None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:17:54.677271 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 857/1704] 10.222.0.26 () {54 vars in 912 bytes} [Thu Jul 14 09:17:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:17:54.687244 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfb0e14-d6a2-40d1-a437-5d0fc9c3beae None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:17:54.687945 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfb0e14-d6a2-40d1-a437-5d0fc9c3beae None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:17:54.688418 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfb0e14-d6a2-40d1-a437-5d0fc9c3beae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:17:54.707939 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e1e46f-1502-46b1-a941-8026673a0747 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:17:54.708230 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e1e46f-1502-46b1-a941-8026673a0747 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:17:54.708706 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e1e46f-1502-46b1-a941-8026673a0747 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:17:54.733769 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:17:54.748010 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-ccfb0e14-d6a2-40d1-a437-5d0fc9c3beae None None] MFA Rules not processed for user `952911c492444ffea4b211a24b564117`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:17:54.749421 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-ccfb0e14-d6a2-40d1-a437-5d0fc9c3beae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:17:54.764324 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 848/1705] 10.222.0.26 () {58 vars in 1009 bytes} [Thu Jul 14 09:17:54 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 09:17:54.774727 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-59e1e46f-1502-46b1-a941-8026673a0747 None None] MFA Rules not processed for user `929b981098ba4ee18b306e21bfb18424`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:17:54.777739 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-59e1e46f-1502-46b1-a941-8026673a0747 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:17:54.812248 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfb0e14-d6a2-40d1-a437-5d0fc9c3beae None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:17:54.823692 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfb0e14-d6a2-40d1-a437-5d0fc9c3beae None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:17:54.834920 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f992f5-3525-4054-824c-5d671bf1448e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '952911c492444ffea4b211a24b564117', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:17:54.836341 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f992f5-3525-4054-824c-5d671bf1448e service neutron] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:17:54.836913 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f992f5-3525-4054-824c-5d671bf1448e service neutron] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:17:54.839549 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f992f5-3525-4054-824c-5d671bf1448e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:17:54.853061 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f992f5-3525-4054-824c-5d671bf1448e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:17:54.855054 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f992f5-3525-4054-824c-5d671bf1448e service neutron] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:17:54.865721 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 849/1706] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:17:54 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:17:54.950254 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 858/1707] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:17:54 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:17:55.020096 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f992f5-3525-4054-824c-5d671bf1448e service neutron] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:17:55.021811 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-16f992f5-3525-4054-824c-5d671bf1448e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:17:55.052664 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f992f5-3525-4054-824c-5d671bf1448e service neutron] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:17:55.060060 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6bea32-bf34-4a3a-8d56-f004cd3521bb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '538fff3c5a2b487eb44ae7c1a1198102', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:17:55.060775 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6bea32-bf34-4a3a-8d56-f004cd3521bb service nova] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:17:55.061061 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6bea32-bf34-4a3a-8d56-f004cd3521bb service nova] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:17:55.061340 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6bea32-bf34-4a3a-8d56-f004cd3521bb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:17:55.070635 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6bea32-bf34-4a3a-8d56-f004cd3521bb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:17:55.074087 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6bea32-bf34-4a3a-8d56-f004cd3521bb service nova] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:17:55.100171 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 850/1708] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:17:55 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:17:55.207722 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e263b2-ee45-4aa7-bce6-3766096cb3f7 None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:17:55.208018 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e263b2-ee45-4aa7-bce6-3766096cb3f7 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:17:55.208351 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e263b2-ee45-4aa7-bce6-3766096cb3f7 None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:17:55.210560 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 859/1709] 10.222.0.26 () {54 vars in 960 bytes} [Thu Jul 14 09:17:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:17:55.224158 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5792ba-0263-4eab-a93f-26327772030f None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:17:55.224450 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5792ba-0263-4eab-a93f-26327772030f None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:17:55.224784 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5792ba-0263-4eab-a93f-26327772030f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:17:55.259288 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-6d5792ba-0263-4eab-a93f-26327772030f None None] MFA Rules not processed for user `355684f52e074ad9bfa7fd90bcae6caf`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:17:55.264089 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-6d5792ba-0263-4eab-a93f-26327772030f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:17:55.424259 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 851/1710] 10.222.0.26 () {58 vars in 1057 bytes} [Thu Jul 14 09:17:55 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:17:55.434187 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e263b2-ee45-4aa7-bce6-3766096cb3f7 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:17:55.457050 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e263b2-ee45-4aa7-bce6-3766096cb3f7 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:17:55.479759 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6d35d2-f541-4e8c-ae53-e2a2b888703d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '355684f52e074ad9bfa7fd90bcae6caf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:17:55.481397 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6d35d2-f541-4e8c-ae53-e2a2b888703d service placement] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:17:55.481869 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6d35d2-f541-4e8c-ae53-e2a2b888703d service placement] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:17:55.482945 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6d35d2-f541-4e8c-ae53-e2a2b888703d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:17:55.485643 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-2a6d35d2-f541-4e8c-ae53-e2a2b888703d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:17:55.515800 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6d35d2-f541-4e8c-ae53-e2a2b888703d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:17:55.519744 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6d35d2-f541-4e8c-ae53-e2a2b888703d service placement] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:17:55.565932 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 860/1711] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:17:55 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:17:56.698568 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77fb04a-ce61-49f5-80ea-ed3f958d0373 None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:17:56.699311 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77fb04a-ce61-49f5-80ea-ed3f958d0373 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:17:56.699311 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77fb04a-ce61-49f5-80ea-ed3f958d0373 None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:17:56.701792 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662daf2a-a83e-4fad-9a11-9fc9e60807d1 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:17:56.702264 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662daf2a-a83e-4fad-9a11-9fc9e60807d1 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:17:56.702457 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662daf2a-a83e-4fad-9a11-9fc9e60807d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:17:56.704390 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 861/1712] 10.222.0.26 () {54 vars in 921 bytes} [Thu Jul 14 09:17:56 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:17:56.765416 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f717200f-b1cc-4799-91ba-38b59c82adbc None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:17:56.765705 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f717200f-b1cc-4799-91ba-38b59c82adbc None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:17:56.765980 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f717200f-b1cc-4799-91ba-38b59c82adbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:17:56.799636 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-662daf2a-a83e-4fad-9a11-9fc9e60807d1 None None] MFA Rules not processed for user `bf084512b5bd4444adfd200e44064741`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:17:56.800853 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-662daf2a-a83e-4fad-9a11-9fc9e60807d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:17:56.836412 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-f717200f-b1cc-4799-91ba-38b59c82adbc None None] MFA Rules not processed for user `355684f52e074ad9bfa7fd90bcae6caf`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:17:56.838659 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-f717200f-b1cc-4799-91ba-38b59c82adbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:17:56.878399 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 862/1713] 10.222.0.26 () {58 vars in 1018 bytes} [Thu Jul 14 09:17:56 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:17:56.882074 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b65205-b899-4824-818b-2dc3ae388e68 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:17:56.883709 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b65205-b899-4824-818b-2dc3ae388e68 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:17:56.885045 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b65205-b899-4824-818b-2dc3ae388e68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:17:56.915435 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-c9b65205-b899-4824-818b-2dc3ae388e68 None None] MFA Rules not processed for user `18377cf2b2d14cb0b4cbda6ec720ed28`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:17:56.917356 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-c9b65205-b899-4824-818b-2dc3ae388e68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:17:57.032084 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 863/1714] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:17:56 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:17:57.034095 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b65205-b899-4824-818b-2dc3ae388e68 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:17:57.043455 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 852/1715] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:17:56 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 347 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:17:57.047320 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662daf2a-a83e-4fad-9a11-9fc9e60807d1 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:17:57.073624 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662daf2a-a83e-4fad-9a11-9fc9e60807d1 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:17:57.076018 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b65205-b899-4824-818b-2dc3ae388e68 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:17:57.080585 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee88464-4391-4cc0-b000-09970a516f3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:17:57.081230 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee88464-4391-4cc0-b000-09970a516f3a admin admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:17:57.081695 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee88464-4391-4cc0-b000-09970a516f3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:17:57.082123 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee88464-4391-4cc0-b000-09970a516f3a admin admin] PATH_INFO: `/v3/projects` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:17:57.083158 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee88464-4391-4cc0-b000-09970a516f3a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:17:57.085476 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee88464-4391-4cc0-b000-09970a516f3a admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:17:57.094470 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82a57b4-4808-442e-868b-5fa0b56a2c82 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '355684f52e074ad9bfa7fd90bcae6caf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:17:57.095293 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82a57b4-4808-442e-868b-5fa0b56a2c82 service placement] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:17:57.099402 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82a57b4-4808-442e-868b-5fa0b56a2c82 service placement] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:17:57.100446 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82a57b4-4808-442e-868b-5fa0b56a2c82 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:17:57.101649 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:17:57.139295 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 853/1716] 10.222.0.26 () {60 vars in 1152 bytes} [Thu Jul 14 09:17:57 2022] POST /identity/v3/projects => generated 373 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 09:17:57.142002 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82a57b4-4808-442e-868b-5fa0b56a2c82 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:17:57.143601 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee88464-4391-4cc0-b000-09970a516f3a admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:17:57.148566 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82a57b4-4808-442e-868b-5fa0b56a2c82 service placement] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:17:57.185381 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 864/1717] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:17:57 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:17:57.516342 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee88464-4391-4cc0-b000-09970a516f3a admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:17:57.563808 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ec9fba-21e3-497d-b044-9a2c4c619b85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:17:57.564524 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ec9fba-21e3-497d-b044-9a2c4c619b85 admin admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:17:57.564813 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ec9fba-21e3-497d-b044-9a2c4c619b85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:17:57.565095 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ec9fba-21e3-497d-b044-9a2c4c619b85 admin admin] PATH_INFO: `/v3/users` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:17:57.566325 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ec9fba-21e3-497d-b044-9a2c4c619b85 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:17:57.568351 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ec9fba-21e3-497d-b044-9a2c4c619b85 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:17:57.638933 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 854/1718] 10.222.0.26 () {60 vars in 1143 bytes} [Thu Jul 14 09:17:57 2022] POST /identity/v3/users => generated 293 bytes in 496 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 09:17:57.645024 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82a57b4-4808-442e-868b-5fa0b56a2c82 service placement] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:17:57.662218 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82a57b4-4808-442e-868b-5fa0b56a2c82 service placement] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:17:57.676334 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6005a779-f086-476c-9741-ded1b26525ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:17:57.677782 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6005a779-f086-476c-9741-ded1b26525ff admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:17:57.678220 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6005a779-f086-476c-9741-ded1b26525ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:17:57.678494 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6005a779-f086-476c-9741-ded1b26525ff admin admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:17:57.680461 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6005a779-f086-476c-9741-ded1b26525ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:17:57.683345 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6005a779-f086-476c-9741-ded1b26525ff admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:17:57.736166 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 865/1719] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jul 14 09:17:57 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 09:17:57.739170 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ec9fba-21e3-497d-b044-9a2c4c619b85 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:17:57.758893 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ec9fba-21e3-497d-b044-9a2c4c619b85 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:17:57.775531 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac46b17c-b08a-4dc9-a7b5-6b28ae50a91a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:17:57.777459 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac46b17c-b08a-4dc9-a7b5-6b28ae50a91a admin admin] REQUEST_METHOD: `PUT` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:17:57.778113 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac46b17c-b08a-4dc9-a7b5-6b28ae50a91a admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:17:57.778746 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac46b17c-b08a-4dc9-a7b5-6b28ae50a91a admin admin] PATH_INFO: `/v3/projects/ac934a31c1364a4f8231d8eaa11044eb/users/27b9db8dc2e34872b2a39bb4d527c5e8/roles/7b549d58d1724395b343609d895c80d0` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:17:57.812599 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac46b17c-b08a-4dc9-a7b5-6b28ae50a91a admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac934a31c1364a4f8231d8eaa11044eb, user_id=27b9db8dc2e34872b2a39bb4d527c5e8, role_id=7b549d58d1724395b343609d895c80d0)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:17:57.819333 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac46b17c-b08a-4dc9-a7b5-6b28ae50a91a admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:17:57.874624 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 855/1720] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jul 14 09:17:57 2022] PUT /identity/v3/projects/ac934a31c1364a4f8231d8eaa11044eb/users/27b9db8dc2e34872b2a39bb4d527c5e8/roles/7b549d58d1724395b343609d895c80d0 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:17:57.878916 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6005a779-f086-476c-9741-ded1b26525ff admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:17:57.895736 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6005a779-f086-476c-9741-ded1b26525ff admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:17:58.047397 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:17:58.069958 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b53bed-d0ec-425a-a1a7-1c2e9016c5fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:17:58.070746 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b53bed-d0ec-425a-a1a7-1c2e9016c5fe admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:17:58.073843 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b53bed-d0ec-425a-a1a7-1c2e9016c5fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:17:58.078202 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b53bed-d0ec-425a-a1a7-1c2e9016c5fe admin admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:17:58.079225 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b53bed-d0ec-425a-a1a7-1c2e9016c5fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:17:58.082546 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b53bed-d0ec-425a-a1a7-1c2e9016c5fe admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:17:58.113079 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 866/1721] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jul 14 09:17:57 2022] GET /identity/v3/roles => generated 1420 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 09:17:58.115950 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac46b17c-b08a-4dc9-a7b5-6b28ae50a91a admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:17:58.129851 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac46b17c-b08a-4dc9-a7b5-6b28ae50a91a admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:17:58.150050 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccaf41e1-b726-4d13-a0cf-f6680ea7cbe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:17:58.151105 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaf41e1-b726-4d13-a0cf-f6680ea7cbe7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:17:58.151694 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaf41e1-b726-4d13-a0cf-f6680ea7cbe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:17:58.152104 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaf41e1-b726-4d13-a0cf-f6680ea7cbe7 admin admin] PATH_INFO: `/v3/projects/ac934a31c1364a4f8231d8eaa11044eb/users/27b9db8dc2e34872b2a39bb4d527c5e8/roles/7b549d58d1724395b343609d895c80d0` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:17:58.156385 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccaf41e1-b726-4d13-a0cf-f6680ea7cbe7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac934a31c1364a4f8231d8eaa11044eb, user_id=27b9db8dc2e34872b2a39bb4d527c5e8, role_id=7b549d58d1724395b343609d895c80d0)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:17:58.163882 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccaf41e1-b726-4d13-a0cf-f6680ea7cbe7 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:17:58.194669 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 856/1722] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jul 14 09:17:58 2022] PUT /identity/v3/projects/ac934a31c1364a4f8231d8eaa11044eb/users/27b9db8dc2e34872b2a39bb4d527c5e8/roles/7b549d58d1724395b343609d895c80d0 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:17:59.193097 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b53bed-d0ec-425a-a1a7-1c2e9016c5fe admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:17:59.198465 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-90b53bed-d0ec-425a-a1a7-1c2e9016c5fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:17:59.274780 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b53bed-d0ec-425a-a1a7-1c2e9016c5fe admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:17:59.574419 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e443b03-0fa5-47be-992e-63c3fc3cdf5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:17:59.576160 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e443b03-0fa5-47be-992e-63c3fc3cdf5f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:17:59.576658 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e443b03-0fa5-47be-992e-63c3fc3cdf5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:17:59.577086 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e443b03-0fa5-47be-992e-63c3fc3cdf5f admin admin] PATH_INFO: `/v3/users/b212293150a44f5b89100607e2ed3dd2` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:17:59.579720 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e443b03-0fa5-47be-992e-63c3fc3cdf5f admin admin] RBAC: Authorizing `identity:delete_user(user_id=b212293150a44f5b89100607e2ed3dd2)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:17:59.583897 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e443b03-0fa5-47be-992e-63c3fc3cdf5f admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:17:59.774514 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-8e443b03-0fa5-47be-992e-63c3fc3cdf5f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b212293150a44f5b89100607e2ed3dd2'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:17:59.776648 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-8e443b03-0fa5-47be-992e-63c3fc3cdf5f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b212293150a44f5b89100607e2ed3dd2'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:17:59.800045 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-8e443b03-0fa5-47be-992e-63c3fc3cdf5f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b212293150a44f5b89100607e2ed3dd2'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:17:59.816276 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-8e443b03-0fa5-47be-992e-63c3fc3cdf5f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b212293150a44f5b89100607e2ed3dd2'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:17:59.817099 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-8e443b03-0fa5-47be-992e-63c3fc3cdf5f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b212293150a44f5b89100607e2ed3dd2'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:17:59.870829 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 867/1723] 10.222.0.26 () {58 vars in 1223 bytes} [Thu Jul 14 09:17:59 2022] DELETE /identity/v3/users/b212293150a44f5b89100607e2ed3dd2 => generated 0 bytes in 678 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:18:00.554062 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccaf41e1-b726-4d13-a0cf-f6680ea7cbe7 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:00.560651 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-ccaf41e1-b726-4d13-a0cf-f6680ea7cbe7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:18:00.583783 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccaf41e1-b726-4d13-a0cf-f6680ea7cbe7 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:00.784591 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a0382d-1e76-40dd-9298-20872dacb694 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:00.785479 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a0382d-1e76-40dd-9298-20872dacb694 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:00.785761 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a0382d-1e76-40dd-9298-20872dacb694 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:00.786037 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a0382d-1e76-40dd-9298-20872dacb694 admin admin] PATH_INFO: `/v3/projects/63da73ee220842e89a71f8436dffdd06` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:00.788321 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a0382d-1e76-40dd-9298-20872dacb694 admin admin] RBAC: Authorizing `identity:delete_project(project_id=63da73ee220842e89a71f8436dffdd06)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:00.789356 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: WARNING py.warnings [None req-18a0382d-1e76-40dd-9298-20872dacb694 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 14 09:18:00.789356 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: warnings.warn(msg) Jul 14 09:18:00.789356 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: Jul 14 09:18:00.795811 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a0382d-1e76-40dd-9298-20872dacb694 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:00.933233 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=91,keystone:INSERT=6,keystone:DELETE=3 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:18:00.958545 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-18a0382d-1e76-40dd-9298-20872dacb694 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '63da73ee220842e89a71f8436dffdd06', 'request_id': 'req-18a0382d-1e76-40dd-9298-20872dacb694'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:01.012588 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-18a0382d-1e76-40dd-9298-20872dacb694 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '63da73ee220842e89a71f8436dffdd06', 'request_id': 'req-18a0382d-1e76-40dd-9298-20872dacb694'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:01.040097 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-18a0382d-1e76-40dd-9298-20872dacb694 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '63da73ee220842e89a71f8436dffdd06', 'request_id': 'req-18a0382d-1e76-40dd-9298-20872dacb694'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:01.077600 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-18a0382d-1e76-40dd-9298-20872dacb694 admin admin] The token cache is being invalidate because project 63da73ee220842e89a71f8436dffdd06 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80381) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 09:18:01.077894 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-18a0382d-1e76-40dd-9298-20872dacb694 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:01.081942 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-18a0382d-1e76-40dd-9298-20872dacb694 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:01.086665 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 857/1724] 10.222.0.26 () {58 vars in 1232 bytes} [Thu Jul 14 09:18:00 2022] DELETE /identity/v3/projects/63da73ee220842e89a71f8436dffdd06 => generated 0 bytes in 533 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:18:01.091181 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e443b03-0fa5-47be-992e-63c3fc3cdf5f admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:01.112178 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e443b03-0fa5-47be-992e-63c3fc3cdf5f admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:01.188145 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec8a00c-5bd3-4952-91be-a58ac7c6ca03 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:01.188447 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec8a00c-5bd3-4952-91be-a58ac7c6ca03 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:01.188738 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec8a00c-5bd3-4952-91be-a58ac7c6ca03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:01.217916 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-eec8a00c-5bd3-4952-91be-a58ac7c6ca03 None None] MFA Rules not processed for user `a2ab766e96024b899a01c540d9bd1322`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:18:01.219338 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-eec8a00c-5bd3-4952-91be-a58ac7c6ca03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:18:01.309514 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713e84ba-8f04-45c1-87b2-cf6e7ce84ed4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:01.311341 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713e84ba-8f04-45c1-87b2-cf6e7ce84ed4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:01.311794 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713e84ba-8f04-45c1-87b2-cf6e7ce84ed4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:01.312299 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713e84ba-8f04-45c1-87b2-cf6e7ce84ed4 admin admin] PATH_INFO: `/v3/users/929b981098ba4ee18b306e21bfb18424` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:01.315874 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713e84ba-8f04-45c1-87b2-cf6e7ce84ed4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=929b981098ba4ee18b306e21bfb18424)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:01.321649 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713e84ba-8f04-45c1-87b2-cf6e7ce84ed4 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:01.342849 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:18:01.397512 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 858/1725] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:18:01 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:18:01.461454 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-713e84ba-8f04-45c1-87b2-cf6e7ce84ed4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '929b981098ba4ee18b306e21bfb18424'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:01.475866 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-713e84ba-8f04-45c1-87b2-cf6e7ce84ed4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '929b981098ba4ee18b306e21bfb18424'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:01.528689 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-713e84ba-8f04-45c1-87b2-cf6e7ce84ed4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '929b981098ba4ee18b306e21bfb18424'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:01.553882 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-713e84ba-8f04-45c1-87b2-cf6e7ce84ed4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '929b981098ba4ee18b306e21bfb18424'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:01.554839 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-713e84ba-8f04-45c1-87b2-cf6e7ce84ed4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '929b981098ba4ee18b306e21bfb18424'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:01.574093 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 868/1726] 10.222.0.26 () {58 vars in 1223 bytes} [Thu Jul 14 09:18:01 2022] DELETE /identity/v3/users/929b981098ba4ee18b306e21bfb18424 => generated 0 bytes in 483 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:18:01.995700 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81353261-2152-41be-803c-74f7d3cf2d1e None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:01.996164 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81353261-2152-41be-803c-74f7d3cf2d1e None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:01.996362 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81353261-2152-41be-803c-74f7d3cf2d1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:02.118328 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-81353261-2152-41be-803c-74f7d3cf2d1e None None] MFA Rules not processed for user `7d6267ede8d64e2abe1f4058a9797fb7`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:18:02.121050 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-81353261-2152-41be-803c-74f7d3cf2d1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:18:02.314636 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 859/1727] 10.222.0.26 () {58 vars in 1009 bytes} [Thu Jul 14 09:18:01 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 320 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:18:02.473346 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af52900-03bd-4c84-a64c-ad1043bc07db None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:02.473866 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af52900-03bd-4c84-a64c-ad1043bc07db None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:02.473953 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af52900-03bd-4c84-a64c-ad1043bc07db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:02.560965 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-8af52900-03bd-4c84-a64c-ad1043bc07db None None] MFA Rules not processed for user `27b9db8dc2e34872b2a39bb4d527c5e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:18:02.563613 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-8af52900-03bd-4c84-a64c-ad1043bc07db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:18:02.718844 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 869/1728] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:18:02 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:18:02.731299 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81353261-2152-41be-803c-74f7d3cf2d1e None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:02.732926 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-81353261-2152-41be-803c-74f7d3cf2d1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:18:02.746923 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af52900-03bd-4c84-a64c-ad1043bc07db None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:02.755880 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-8af52900-03bd-4c84-a64c-ad1043bc07db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:18:02.909880 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81353261-2152-41be-803c-74f7d3cf2d1e None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:02.920069 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c47c151-58bd-43af-93cd-fd89000151b0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e76f1952a5b146808844b9f2b7395035', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:02.922040 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c47c151-58bd-43af-93cd-fd89000151b0 service cinder] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:02.923082 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c47c151-58bd-43af-93cd-fd89000151b0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:02.923518 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c47c151-58bd-43af-93cd-fd89000151b0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:02.929180 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-5c47c151-58bd-43af-93cd-fd89000151b0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:18:02.942176 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:DELETE=4,keystone:SELECT=95,keystone:INSERT=1 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:18:02.967711 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:18:02.999974 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af52900-03bd-4c84-a64c-ad1043bc07db None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:03.008964 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d93436-8966-4011-8c8c-704923781760 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5a43c80514b4a13af03c7f8ebcf7a5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:03.010361 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d93436-8966-4011-8c8c-704923781760 service swift] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:03.010361 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d93436-8966-4011-8c8c-704923781760 service swift] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:03.010561 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d93436-8966-4011-8c8c-704923781760 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:03.030148 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d93436-8966-4011-8c8c-704923781760 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:03.032143 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d93436-8966-4011-8c8c-704923781760 service swift] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:03.047295 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 870/1729] 10.222.0.26 () {60 vars in 1384 bytes} [Thu Jul 14 09:18:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 14 09:18:03.064303 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c47c151-58bd-43af-93cd-fd89000151b0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:03.066524 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c47c151-58bd-43af-93cd-fd89000151b0 service cinder] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:03.120892 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 860/1730] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:18:02 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:18:03.397593 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d93436-8966-4011-8c8c-704923781760 service swift] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:03.399498 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-07d93436-8966-4011-8c8c-704923781760 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:18:03.564185 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d93436-8966-4011-8c8c-704923781760 service swift] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:03.578779 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39873ef3-be4d-4d57-a2d7-e0ccbca28382 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:03.579957 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39873ef3-be4d-4d57-a2d7-e0ccbca28382 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:03.580722 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39873ef3-be4d-4d57-a2d7-e0ccbca28382 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:03.582589 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39873ef3-be4d-4d57-a2d7-e0ccbca28382 admin admin] PATH_INFO: `/v3/projects/d294bb86d04e4d5386e441ed183fa9ae` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:03.585941 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39873ef3-be4d-4d57-a2d7-e0ccbca28382 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d294bb86d04e4d5386e441ed183fa9ae)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:03.598111 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39873ef3-be4d-4d57-a2d7-e0ccbca28382 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:03.748645 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-39873ef3-be4d-4d57-a2d7-e0ccbca28382 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd294bb86d04e4d5386e441ed183fa9ae', 'request_id': 'req-39873ef3-be4d-4d57-a2d7-e0ccbca28382'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:03.758096 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-39873ef3-be4d-4d57-a2d7-e0ccbca28382 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd294bb86d04e4d5386e441ed183fa9ae', 'request_id': 'req-39873ef3-be4d-4d57-a2d7-e0ccbca28382'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:03.765664 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-39873ef3-be4d-4d57-a2d7-e0ccbca28382 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd294bb86d04e4d5386e441ed183fa9ae', 'request_id': 'req-39873ef3-be4d-4d57-a2d7-e0ccbca28382'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:03.825377 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-39873ef3-be4d-4d57-a2d7-e0ccbca28382 admin admin] The token cache is being invalidate because project d294bb86d04e4d5386e441ed183fa9ae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80382) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 09:18:03.825933 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-39873ef3-be4d-4d57-a2d7-e0ccbca28382 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:03.831480 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-39873ef3-be4d-4d57-a2d7-e0ccbca28382 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:03.834329 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 871/1731] 10.222.0.26 () {58 vars in 1232 bytes} [Thu Jul 14 09:18:03 2022] DELETE /identity/v3/projects/d294bb86d04e4d5386e441ed183fa9ae => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:18:03.839418 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cab7e4e-f057-42ae-974a-f0377a382df9 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:03.840020 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cab7e4e-f057-42ae-974a-f0377a382df9 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:03.840609 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cab7e4e-f057-42ae-974a-f0377a382df9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:03.878744 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-5cab7e4e-f057-42ae-974a-f0377a382df9 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:18:03.880367 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-5cab7e4e-f057-42ae-974a-f0377a382df9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:18:04.139505 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 861/1732] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:18:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 301 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 09:18:04.152279 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f6d9d0-c00d-4c8e-8b8c-806f8c0a5d67 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:04.152279 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f6d9d0-c00d-4c8e-8b8c-806f8c0a5d67 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:04.152279 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f6d9d0-c00d-4c8e-8b8c-806f8c0a5d67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:04.219172 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-45f6d9d0-c00d-4c8e-8b8c-806f8c0a5d67 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:18:04.224143 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-45f6d9d0-c00d-4c8e-8b8c-806f8c0a5d67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:18:04.252784 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 872/1733] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:18:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:18:04.257839 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cab7e4e-f057-42ae-974a-f0377a382df9 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:04.291063 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cab7e4e-f057-42ae-974a-f0377a382df9 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:04.297516 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef55f4a-27e4-411f-a333-f6bdb20ef78c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:04.298582 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef55f4a-27e4-411f-a333-f6bdb20ef78c admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:04.299151 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef55f4a-27e4-411f-a333-f6bdb20ef78c admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:04.299671 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef55f4a-27e4-411f-a333-f6bdb20ef78c admin admin] PATH_INFO: `/v3/domains` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:04.300738 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef55f4a-27e4-411f-a333-f6bdb20ef78c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:18:04.301812 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef55f4a-27e4-411f-a333-f6bdb20ef78c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:04.304646 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef55f4a-27e4-411f-a333-f6bdb20ef78c admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:04.331989 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 862/1734] 10.222.0.26 () {58 vars in 1152 bytes} [Thu Jul 14 09:18:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:18:04.334403 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f6d9d0-c00d-4c8e-8b8c-806f8c0a5d67 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:04.349395 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f6d9d0-c00d-4c8e-8b8c-806f8c0a5d67 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:04.367069 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26514eb-83b8-4fd7-81b1-f479f815d4ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:04.369597 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26514eb-83b8-4fd7-81b1-f479f815d4ca admin admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:04.369889 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26514eb-83b8-4fd7-81b1-f479f815d4ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:04.370165 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26514eb-83b8-4fd7-81b1-f479f815d4ca admin admin] PATH_INFO: `/v3/projects` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:04.376206 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26514eb-83b8-4fd7-81b1-f479f815d4ca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:04.378172 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26514eb-83b8-4fd7-81b1-f479f815d4ca admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:04.441956 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 873/1735] 10.222.0.26 () {60 vars in 1152 bytes} [Thu Jul 14 09:18:04 2022] POST /identity/v3/projects => generated 385 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 09:18:04.446060 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef55f4a-27e4-411f-a333-f6bdb20ef78c admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:04.461489 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef55f4a-27e4-411f-a333-f6bdb20ef78c admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:04.605434 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a9eb9a-2159-4e73-8033-d0b426ed3012 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:04.606649 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a9eb9a-2159-4e73-8033-d0b426ed3012 admin admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:04.608814 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a9eb9a-2159-4e73-8033-d0b426ed3012 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:04.609187 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a9eb9a-2159-4e73-8033-d0b426ed3012 admin admin] PATH_INFO: `/v3/users` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:04.611180 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a9eb9a-2159-4e73-8033-d0b426ed3012 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:04.616404 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a9eb9a-2159-4e73-8033-d0b426ed3012 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:04.656224 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 863/1736] 10.222.0.26 () {60 vars in 1143 bytes} [Thu Jul 14 09:18:04 2022] POST /identity/v3/users => generated 293 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 09:18:04.660337 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26514eb-83b8-4fd7-81b1-f479f815d4ca admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:04.671522 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26514eb-83b8-4fd7-81b1-f479f815d4ca admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:04.693703 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc274aea-dded-42c4-878f-c49f3c4a8ded admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:04.694413 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc274aea-dded-42c4-878f-c49f3c4a8ded admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:04.694699 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc274aea-dded-42c4-878f-c49f3c4a8ded admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:04.694975 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc274aea-dded-42c4-878f-c49f3c4a8ded admin admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:04.695757 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc274aea-dded-42c4-878f-c49f3c4a8ded admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:04.702675 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc274aea-dded-42c4-878f-c49f3c4a8ded admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:04.739240 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 874/1737] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jul 14 09:18:04 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 09:18:04.740800 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a9eb9a-2159-4e73-8033-d0b426ed3012 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:04.752561 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a9eb9a-2159-4e73-8033-d0b426ed3012 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:04.757329 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d28299-8735-4d51-84a0-d3d2fc7658ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:04.758024 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d28299-8735-4d51-84a0-d3d2fc7658ac admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:04.758330 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d28299-8735-4d51-84a0-d3d2fc7658ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:04.758573 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d28299-8735-4d51-84a0-d3d2fc7658ac admin admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:04.759353 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d28299-8735-4d51-84a0-d3d2fc7658ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:04.761445 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d28299-8735-4d51-84a0-d3d2fc7658ac admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:04.798867 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc274aea-dded-42c4-878f-c49f3c4a8ded admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:04.799939 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 864/1738] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jul 14 09:18:04 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 09:18:04.810797 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc274aea-dded-42c4-878f-c49f3c4a8ded admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:04.817148 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3c8897-1bc4-473f-8c96-9693b003b1fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:04.818163 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3c8897-1bc4-473f-8c96-9693b003b1fd admin admin] REQUEST_METHOD: `PUT` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:04.818613 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3c8897-1bc4-473f-8c96-9693b003b1fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:04.818978 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3c8897-1bc4-473f-8c96-9693b003b1fd admin admin] PATH_INFO: `/v3/projects/43cf74a97d964c4f98c182a37573ebb3/users/40699e94f2434942927d316e54ecb79a/roles/d1691322ce6a4edab7ce722edf0f2988` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:04.842022 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3c8897-1bc4-473f-8c96-9693b003b1fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=43cf74a97d964c4f98c182a37573ebb3, user_id=40699e94f2434942927d316e54ecb79a, role_id=d1691322ce6a4edab7ce722edf0f2988)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:04.845445 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3c8897-1bc4-473f-8c96-9693b003b1fd admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:04.872214 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 875/1739] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jul 14 09:18:04 2022] PUT /identity/v3/projects/43cf74a97d964c4f98c182a37573ebb3/users/40699e94f2434942927d316e54ecb79a/roles/d1691322ce6a4edab7ce722edf0f2988 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 09:18:04.882802 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cdf802-54e1-4903-849e-add49a96ac21 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:04.883140 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cdf802-54e1-4903-849e-add49a96ac21 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:04.883450 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cdf802-54e1-4903-849e-add49a96ac21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:04.908581 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:18:04.920491 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-55cdf802-54e1-4903-849e-add49a96ac21 None None] MFA Rules not processed for user `40699e94f2434942927d316e54ecb79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:18:04.923331 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-55cdf802-54e1-4903-849e-add49a96ac21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:18:05.052216 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 865/1740] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:18:04 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:18:05.056888 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3c8897-1bc4-473f-8c96-9693b003b1fd admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:05.067336 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:18:05.162520 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3c8897-1bc4-473f-8c96-9693b003b1fd admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:05.168300 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48889e9-0397-4eae-8823-b59a45dc62f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:05.169308 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48889e9-0397-4eae-8823-b59a45dc62f9 admin admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:05.169594 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48889e9-0397-4eae-8823-b59a45dc62f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:05.169872 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48889e9-0397-4eae-8823-b59a45dc62f9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:05.170592 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48889e9-0397-4eae-8823-b59a45dc62f9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:05.174186 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48889e9-0397-4eae-8823-b59a45dc62f9 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:05.207175 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 876/1741] 10.222.0.26 () {60 vars in 1152 bytes} [Thu Jul 14 09:18:05 2022] POST /identity/v3/projects => generated 385 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 09:18:05.211728 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cdf802-54e1-4903-849e-add49a96ac21 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:05.229573 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cdf802-54e1-4903-849e-add49a96ac21 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:05.326575 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9372e4-f4ba-49b5-8d5e-e0555a21c63b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:05.327466 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9372e4-f4ba-49b5-8d5e-e0555a21c63b admin admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:05.327841 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9372e4-f4ba-49b5-8d5e-e0555a21c63b admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:05.328187 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9372e4-f4ba-49b5-8d5e-e0555a21c63b admin admin] PATH_INFO: `/v3/users` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:05.328976 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9372e4-f4ba-49b5-8d5e-e0555a21c63b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:05.331043 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9372e4-f4ba-49b5-8d5e-e0555a21c63b admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:05.360318 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 866/1742] 10.222.0.26 () {60 vars in 1143 bytes} [Thu Jul 14 09:18:05 2022] POST /identity/v3/users => generated 293 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 09:18:05.364494 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48889e9-0397-4eae-8823-b59a45dc62f9 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:05.377276 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48889e9-0397-4eae-8823-b59a45dc62f9 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:05.399037 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fbfc3c-803b-41b7-93ac-af953235bdc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:05.401209 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fbfc3c-803b-41b7-93ac-af953235bdc2 admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:05.402125 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fbfc3c-803b-41b7-93ac-af953235bdc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:05.402587 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fbfc3c-803b-41b7-93ac-af953235bdc2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:05.403976 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fbfc3c-803b-41b7-93ac-af953235bdc2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:05.406266 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fbfc3c-803b-41b7-93ac-af953235bdc2 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:05.437713 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 877/1743] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jul 14 09:18:05 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 09:18:05.440202 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9372e4-f4ba-49b5-8d5e-e0555a21c63b admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:05.455017 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9372e4-f4ba-49b5-8d5e-e0555a21c63b admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:05.461199 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df75c98-7431-47c1-a8a7-7393a488eff8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:05.462113 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df75c98-7431-47c1-a8a7-7393a488eff8 admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:05.462711 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df75c98-7431-47c1-a8a7-7393a488eff8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:05.463133 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df75c98-7431-47c1-a8a7-7393a488eff8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:05.464060 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df75c98-7431-47c1-a8a7-7393a488eff8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:05.466103 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df75c98-7431-47c1-a8a7-7393a488eff8 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:05.492401 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fbfc3c-803b-41b7-93ac-af953235bdc2 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:05.503703 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fbfc3c-803b-41b7-93ac-af953235bdc2 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:05.503939 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 867/1744] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jul 14 09:18:05 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 09:18:05.514566 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665d676f-630f-4665-8aab-3c2901c70526 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:05.515544 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665d676f-630f-4665-8aab-3c2901c70526 admin admin] REQUEST_METHOD: `PUT` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:05.515832 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665d676f-630f-4665-8aab-3c2901c70526 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:05.516109 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665d676f-630f-4665-8aab-3c2901c70526 admin admin] PATH_INFO: `/v3/projects/9ce4daf557254d97b4b3855f645c4ae9/users/7603d171e1f7402db55ea79b247739b5/roles/d1691322ce6a4edab7ce722edf0f2988` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:05.542194 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665d676f-630f-4665-8aab-3c2901c70526 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ce4daf557254d97b4b3855f645c4ae9, user_id=7603d171e1f7402db55ea79b247739b5, role_id=d1691322ce6a4edab7ce722edf0f2988)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:05.545126 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665d676f-630f-4665-8aab-3c2901c70526 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:05.581282 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e79cde-52d3-4076-bd06-87dcbb4f7235 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:05.581580 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e79cde-52d3-4076-bd06-87dcbb4f7235 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:05.581917 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e79cde-52d3-4076-bd06-87dcbb4f7235 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:05.583331 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 878/1745] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jul 14 09:18:05 2022] PUT /identity/v3/projects/9ce4daf557254d97b4b3855f645c4ae9/users/7603d171e1f7402db55ea79b247739b5/roles/d1691322ce6a4edab7ce722edf0f2988 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:18:05.641606 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-36e79cde-52d3-4076-bd06-87dcbb4f7235 None None] MFA Rules not processed for user `7603d171e1f7402db55ea79b247739b5`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:18:05.643406 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-36e79cde-52d3-4076-bd06-87dcbb4f7235 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:18:05.830517 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 868/1746] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:18:05 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:18:05.834554 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665d676f-630f-4665-8aab-3c2901c70526 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:06.076117 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bb5c6d-a9d0-440a-9488-19501451b4b2 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:06.076400 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bb5c6d-a9d0-440a-9488-19501451b4b2 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:06.076713 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bb5c6d-a9d0-440a-9488-19501451b4b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:06.100362 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665d676f-630f-4665-8aab-3c2901c70526 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:06.114013 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242067a2-46c7-4872-be62-ce448fbf5770 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:06.114660 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-b3bb5c6d-a9d0-440a-9488-19501451b4b2 None None] MFA Rules not processed for user `952911c492444ffea4b211a24b564117`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:18:06.115972 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242067a2-46c7-4872-be62-ce448fbf5770 admin admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:06.118101 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-b3bb5c6d-a9d0-440a-9488-19501451b4b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:18:06.127113 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242067a2-46c7-4872-be62-ce448fbf5770 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:06.127561 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242067a2-46c7-4872-be62-ce448fbf5770 admin admin] PATH_INFO: `/v3/projects` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:06.128731 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242067a2-46c7-4872-be62-ce448fbf5770 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:06.132631 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:18:06.133559 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242067a2-46c7-4872-be62-ce448fbf5770 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:06.213850 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 879/1747] 10.222.0.26 () {60 vars in 1152 bytes} [Thu Jul 14 09:18:05 2022] POST /identity/v3/projects => generated 385 bytes in 374 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 09:18:06.217839 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242067a2-46c7-4872-be62-ce448fbf5770 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:06.285735 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 869/1748] 10.222.0.17 () {58 vars in 1015 bytes} [Thu Jul 14 09:18:06 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:18:06.310601 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bb5c6d-a9d0-440a-9488-19501451b4b2 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:06.313115 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-b3bb5c6d-a9d0-440a-9488-19501451b4b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:18:06.333699 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bb5c6d-a9d0-440a-9488-19501451b4b2 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:06.339000 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:18:06.389125 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242067a2-46c7-4872-be62-ce448fbf5770 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:06.406378 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc96bf5b-e914-4299-bc75-d503e3cc2381 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:06.407092 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc96bf5b-e914-4299-bc75-d503e3cc2381 admin admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:06.407403 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc96bf5b-e914-4299-bc75-d503e3cc2381 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:06.407685 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc96bf5b-e914-4299-bc75-d503e3cc2381 admin admin] PATH_INFO: `/v3/users` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:06.408395 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc96bf5b-e914-4299-bc75-d503e3cc2381 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:06.414706 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc96bf5b-e914-4299-bc75-d503e3cc2381 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:06.496892 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 880/1749] 10.222.0.26 () {60 vars in 1143 bytes} [Thu Jul 14 09:18:06 2022] POST /identity/v3/users => generated 299 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 09:18:06.498059 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc96bf5b-e914-4299-bc75-d503e3cc2381 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:06.499051 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3880e2-668d-407f-97e5-970cac3c952e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '952911c492444ffea4b211a24b564117', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:06.499880 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3880e2-668d-407f-97e5-970cac3c952e service neutron] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:06.500123 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3880e2-668d-407f-97e5-970cac3c952e service neutron] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:06.500407 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3880e2-668d-407f-97e5-970cac3c952e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:06.504973 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-fc96bf5b-e914-4299-bc75-d503e3cc2381 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:18:06.519387 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3880e2-668d-407f-97e5-970cac3c952e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:06.527659 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3880e2-668d-407f-97e5-970cac3c952e service neutron] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:06.529767 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc96bf5b-e914-4299-bc75-d503e3cc2381 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:06.535777 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9182188c-b4c1-438e-94a4-f9e48b65a46a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '952911c492444ffea4b211a24b564117', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:06.536453 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9182188c-b4c1-438e-94a4-f9e48b65a46a service neutron] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:06.536759 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9182188c-b4c1-438e-94a4-f9e48b65a46a service neutron] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:06.537003 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9182188c-b4c1-438e-94a4-f9e48b65a46a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:06.548359 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 870/1750] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:18:06 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:18:06.551330 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3880e2-668d-407f-97e5-970cac3c952e service neutron] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:06.562006 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9182188c-b4c1-438e-94a4-f9e48b65a46a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:06.563829 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9182188c-b4c1-438e-94a4-f9e48b65a46a service neutron] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:06.581399 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3880e2-668d-407f-97e5-970cac3c952e service neutron] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:06.583256 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 881/1751] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:18:06 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:18:06.595483 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3caa8945-948f-4100-8b03-c58ddef9c14e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:06.597033 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caa8945-948f-4100-8b03-c58ddef9c14e admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:06.597821 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caa8945-948f-4100-8b03-c58ddef9c14e admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:06.597821 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caa8945-948f-4100-8b03-c58ddef9c14e admin admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:06.598706 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3caa8945-948f-4100-8b03-c58ddef9c14e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:06.600514 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3caa8945-948f-4100-8b03-c58ddef9c14e admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:06.625667 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 871/1752] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jul 14 09:18:06 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 09:18:06.632743 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9182188c-b4c1-438e-94a4-f9e48b65a46a service neutron] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:06.645897 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9182188c-b4c1-438e-94a4-f9e48b65a46a service neutron] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:06.660168 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a260a53d-5dd1-4596-8869-cb26d887d57c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:06.661089 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a260a53d-5dd1-4596-8869-cb26d887d57c admin admin] REQUEST_METHOD: `PUT` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:06.661367 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a260a53d-5dd1-4596-8869-cb26d887d57c admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:06.661637 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a260a53d-5dd1-4596-8869-cb26d887d57c admin admin] PATH_INFO: `/v3/projects/4fab3c6360fd4339a534f0ed1fb80cec/users/beca41779b324b2782d94d73cb8bba1f/roles/7b549d58d1724395b343609d895c80d0` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:06.682675 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a260a53d-5dd1-4596-8869-cb26d887d57c admin admin] RBAC: Authorizing `identity:create_grant(project_id=4fab3c6360fd4339a534f0ed1fb80cec, user_id=beca41779b324b2782d94d73cb8bba1f, role_id=7b549d58d1724395b343609d895c80d0)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:06.685596 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a260a53d-5dd1-4596-8869-cb26d887d57c admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:06.735017 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 882/1753] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jul 14 09:18:06 2022] PUT /identity/v3/projects/4fab3c6360fd4339a534f0ed1fb80cec/users/beca41779b324b2782d94d73cb8bba1f/roles/7b549d58d1724395b343609d895c80d0 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:18:06.739305 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3caa8945-948f-4100-8b03-c58ddef9c14e admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:06.753316 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3caa8945-948f-4100-8b03-c58ddef9c14e admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:06.915206 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b06ee83-b83e-401a-a196-55380087cdef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:06.916357 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b06ee83-b83e-401a-a196-55380087cdef admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:06.916641 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b06ee83-b83e-401a-a196-55380087cdef admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:06.916914 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b06ee83-b83e-401a-a196-55380087cdef admin admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:06.917807 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b06ee83-b83e-401a-a196-55380087cdef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:06.926063 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b06ee83-b83e-401a-a196-55380087cdef admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:06.955349 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a260a53d-5dd1-4596-8869-cb26d887d57c admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:06.956969 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 872/1754] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jul 14 09:18:06 2022] GET /identity/v3/roles => generated 1420 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 09:18:06.969426 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a260a53d-5dd1-4596-8869-cb26d887d57c admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:06.985034 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7736b511-02a0-4a92-af43-3919a395e2ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:06.985959 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7736b511-02a0-4a92-af43-3919a395e2ad admin admin] REQUEST_METHOD: `PUT` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:06.986236 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7736b511-02a0-4a92-af43-3919a395e2ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:06.986509 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7736b511-02a0-4a92-af43-3919a395e2ad admin admin] PATH_INFO: `/v3/projects/4fab3c6360fd4339a534f0ed1fb80cec/users/beca41779b324b2782d94d73cb8bba1f/roles/7b549d58d1724395b343609d895c80d0` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:06.992624 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7736b511-02a0-4a92-af43-3919a395e2ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=4fab3c6360fd4339a534f0ed1fb80cec, user_id=beca41779b324b2782d94d73cb8bba1f, role_id=7b549d58d1724395b343609d895c80d0)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:06.995329 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7736b511-02a0-4a92-af43-3919a395e2ad admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:07.034469 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 883/1755] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jul 14 09:18:06 2022] PUT /identity/v3/projects/4fab3c6360fd4339a534f0ed1fb80cec/users/beca41779b324b2782d94d73cb8bba1f/roles/7b549d58d1724395b343609d895c80d0 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:18:07.048665 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015848e3-78cc-4dd3-a90b-8e6a79ac3b25 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:07.048999 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015848e3-78cc-4dd3-a90b-8e6a79ac3b25 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:07.049224 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015848e3-78cc-4dd3-a90b-8e6a79ac3b25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:07.112360 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-015848e3-78cc-4dd3-a90b-8e6a79ac3b25 None None] MFA Rules not processed for user `beca41779b324b2782d94d73cb8bba1f`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:18:07.113669 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-015848e3-78cc-4dd3-a90b-8e6a79ac3b25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:18:07.186060 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:18:07.309022 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 873/1756] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:18:07 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:18:07.355002 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7736b511-02a0-4a92-af43-3919a395e2ad admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:07.356776 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-7736b511-02a0-4a92-af43-3919a395e2ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:18:07.584971 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7736b511-02a0-4a92-af43-3919a395e2ad admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:07.599835 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d55d525-5a40-4111-80d5-0f80f895cd63 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e76f1952a5b146808844b9f2b7395035', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:07.600870 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d55d525-5a40-4111-80d5-0f80f895cd63 service cinder] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:07.603141 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d55d525-5a40-4111-80d5-0f80f895cd63 service cinder] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:07.603891 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d55d525-5a40-4111-80d5-0f80f895cd63 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:07.612194 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-9d55d525-5a40-4111-80d5-0f80f895cd63 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:18:07.616390 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015848e3-78cc-4dd3-a90b-8e6a79ac3b25 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:07.618572 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-015848e3-78cc-4dd3-a90b-8e6a79ac3b25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:18:07.642521 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015848e3-78cc-4dd3-a90b-8e6a79ac3b25 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:07.649113 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762a62c0-d113-4351-8746-e2472030c302 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e76f1952a5b146808844b9f2b7395035', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:07.649801 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762a62c0-d113-4351-8746-e2472030c302 service cinder] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:07.650129 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762a62c0-d113-4351-8746-e2472030c302 service cinder] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:07.654466 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762a62c0-d113-4351-8746-e2472030c302 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:07.694265 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762a62c0-d113-4351-8746-e2472030c302 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:07.702789 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762a62c0-d113-4351-8746-e2472030c302 service cinder] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:07.765044 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 874/1757] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:18:07 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:18:07.844284 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d55d525-5a40-4111-80d5-0f80f895cd63 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:07.846536 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d55d525-5a40-4111-80d5-0f80f895cd63 service cinder] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:07.888682 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 884/1758] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:18:07 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 534 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 09:18:09.526203 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762a62c0-d113-4351-8746-e2472030c302 service cinder] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:09.542015 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762a62c0-d113-4351-8746-e2472030c302 service cinder] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:09.561877 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5de386-ec38-44da-b970-f75ecf86e760 None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:09.562516 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5de386-ec38-44da-b970-f75ecf86e760 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:09.563144 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5de386-ec38-44da-b970-f75ecf86e760 None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:09.567219 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 885/1759] 10.222.0.26 () {54 vars in 921 bytes} [Thu Jul 14 09:18:09 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:18:09.580880 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3da55e-0b06-4d24-bf39-881ce3e04690 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e76f1952a5b146808844b9f2b7395035', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:09.584953 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3da55e-0b06-4d24-bf39-881ce3e04690 service cinder] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:09.586864 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3da55e-0b06-4d24-bf39-881ce3e04690 service cinder] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:09.587170 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3da55e-0b06-4d24-bf39-881ce3e04690 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:09.756390 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3569e16e-1ded-48b1-ab77-3d13b1fa692f None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:09.757199 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3569e16e-1ded-48b1-ab77-3d13b1fa692f None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:09.757644 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3569e16e-1ded-48b1-ab77-3d13b1fa692f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:09.801502 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3da55e-0b06-4d24-bf39-881ce3e04690 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:09.803243 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3da55e-0b06-4d24-bf39-881ce3e04690 service cinder] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:09.805069 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-3569e16e-1ded-48b1-ab77-3d13b1fa692f None None] MFA Rules not processed for user `538fff3c5a2b487eb44ae7c1a1198102`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:18:09.814911 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-3569e16e-1ded-48b1-ab77-3d13b1fa692f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:18:09.831623 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 875/1760] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:18:09 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:18:10.016097 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:18:10.250475 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 886/1761] 10.222.0.26 () {58 vars in 1018 bytes} [Thu Jul 14 09:18:09 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 495 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:18:10.266733 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3da55e-0b06-4d24-bf39-881ce3e04690 service cinder] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:10.268834 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-2f3da55e-0b06-4d24-bf39-881ce3e04690 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:18:10.323787 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3da55e-0b06-4d24-bf39-881ce3e04690 service cinder] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:10.350802 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fe770a-35b5-4f8e-b4dc-de45a69da08b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '538fff3c5a2b487eb44ae7c1a1198102', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:10.352083 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fe770a-35b5-4f8e-b4dc-de45a69da08b service nova] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:10.352675 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fe770a-35b5-4f8e-b4dc-de45a69da08b service nova] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:10.353930 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fe770a-35b5-4f8e-b4dc-de45a69da08b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:10.375900 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fe770a-35b5-4f8e-b4dc-de45a69da08b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:10.381167 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fe770a-35b5-4f8e-b4dc-de45a69da08b service nova] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:10.437595 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 876/1762] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:18:10 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:18:12.045462 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54d902e-7b76-4b1a-9ad4-b7c46ab51ccf None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:12.046083 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54d902e-7b76-4b1a-9ad4-b7c46ab51ccf None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:12.046083 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54d902e-7b76-4b1a-9ad4-b7c46ab51ccf None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:12.047984 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 887/1763] 10.222.0.26 () {54 vars in 921 bytes} [Thu Jul 14 09:18:12 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:18:12.054865 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854e4e3d-8350-4102-86e7-cd8da8775a07 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:12.055449 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854e4e3d-8350-4102-86e7-cd8da8775a07 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:12.055449 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854e4e3d-8350-4102-86e7-cd8da8775a07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:12.080275 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-854e4e3d-8350-4102-86e7-cd8da8775a07 None None] MFA Rules not processed for user `538fff3c5a2b487eb44ae7c1a1198102`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:18:12.082675 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-854e4e3d-8350-4102-86e7-cd8da8775a07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:18:12.094937 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 877/1764] 10.222.0.26 () {58 vars in 1018 bytes} [Thu Jul 14 09:18:12 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:18:12.111350 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54d902e-7b76-4b1a-9ad4-b7c46ab51ccf None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:12.130418 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54d902e-7b76-4b1a-9ad4-b7c46ab51ccf None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:12.137807 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a6d31d-aedb-4f7f-9b1f-be51e9f933f8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '538fff3c5a2b487eb44ae7c1a1198102', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:12.138469 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a6d31d-aedb-4f7f-9b1f-be51e9f933f8 service nova] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:12.138750 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a6d31d-aedb-4f7f-9b1f-be51e9f933f8 service nova] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:12.139017 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a6d31d-aedb-4f7f-9b1f-be51e9f933f8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:12.153428 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a6d31d-aedb-4f7f-9b1f-be51e9f933f8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:12.155062 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a6d31d-aedb-4f7f-9b1f-be51e9f933f8 service nova] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:12.166381 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 888/1765] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:18:12 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:18:22.070927 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=56 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:18:22.161832 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=22 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:18:23.176218 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854e4e3d-8350-4102-86e7-cd8da8775a07 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:23.185549 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854e4e3d-8350-4102-86e7-cd8da8775a07 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:23.190197 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6804a1e4-e61c-41cd-8c5a-bd214c40c363 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e76f1952a5b146808844b9f2b7395035', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:23.190994 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6804a1e4-e61c-41cd-8c5a-bd214c40c363 service cinder] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:23.191113 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6804a1e4-e61c-41cd-8c5a-bd214c40c363 service cinder] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:23.191448 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6804a1e4-e61c-41cd-8c5a-bd214c40c363 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:23.260435 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6804a1e4-e61c-41cd-8c5a-bd214c40c363 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:23.261946 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6804a1e4-e61c-41cd-8c5a-bd214c40c363 service cinder] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:23.271799 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 878/1766] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:18:23 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:18:24.193718 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a6d31d-aedb-4f7f-9b1f-be51e9f933f8 service nova] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:24.195393 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-e9a6d31d-aedb-4f7f-9b1f-be51e9f933f8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:18:24.267367 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a6d31d-aedb-4f7f-9b1f-be51e9f933f8 service nova] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:24.272637 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65af930e-b680-46d0-81cc-11ea1a26ac65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:24.273534 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65af930e-b680-46d0-81cc-11ea1a26ac65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:24.273814 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65af930e-b680-46d0-81cc-11ea1a26ac65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:24.274086 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65af930e-b680-46d0-81cc-11ea1a26ac65 admin admin] PATH_INFO: `/v3/users/903e43d8e5fa473c9042bf8eeb99b2ba` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:24.275329 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65af930e-b680-46d0-81cc-11ea1a26ac65 admin admin] RBAC: Authorizing `identity:delete_user(user_id=903e43d8e5fa473c9042bf8eeb99b2ba)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:24.277449 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65af930e-b680-46d0-81cc-11ea1a26ac65 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:24.342996 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-65af930e-b680-46d0-81cc-11ea1a26ac65 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '903e43d8e5fa473c9042bf8eeb99b2ba'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:24.343766 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-65af930e-b680-46d0-81cc-11ea1a26ac65 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '903e43d8e5fa473c9042bf8eeb99b2ba'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:24.358523 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-65af930e-b680-46d0-81cc-11ea1a26ac65 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '903e43d8e5fa473c9042bf8eeb99b2ba'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:24.367104 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-65af930e-b680-46d0-81cc-11ea1a26ac65 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '903e43d8e5fa473c9042bf8eeb99b2ba'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:24.367835 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-65af930e-b680-46d0-81cc-11ea1a26ac65 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '903e43d8e5fa473c9042bf8eeb99b2ba'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:24.385239 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 889/1767] 10.222.0.26 () {58 vars in 1223 bytes} [Thu Jul 14 09:18:24 2022] DELETE /identity/v3/users/903e43d8e5fa473c9042bf8eeb99b2ba => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:18:24.389949 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17922374-a207-4e63-9d30-499d81e4b8a4 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:24.390950 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17922374-a207-4e63-9d30-499d81e4b8a4 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:24.390950 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17922374-a207-4e63-9d30-499d81e4b8a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:24.409730 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-17922374-a207-4e63-9d30-499d81e4b8a4 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:18:24.410923 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-17922374-a207-4e63-9d30-499d81e4b8a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:18:24.485729 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 879/1768] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:18:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:18:24.509024 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65af930e-b680-46d0-81cc-11ea1a26ac65 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:24.510830 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-65af930e-b680-46d0-81cc-11ea1a26ac65 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:18:24.624735 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65af930e-b680-46d0-81cc-11ea1a26ac65 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:24.630390 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e0232c-93f5-451c-8ed2-92b25828972a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '952911c492444ffea4b211a24b564117', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:24.631904 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e0232c-93f5-451c-8ed2-92b25828972a service neutron] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:24.631904 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e0232c-93f5-451c-8ed2-92b25828972a service neutron] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:24.632102 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e0232c-93f5-451c-8ed2-92b25828972a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:24.644517 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e0232c-93f5-451c-8ed2-92b25828972a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:24.646503 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e0232c-93f5-451c-8ed2-92b25828972a service neutron] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:24.657279 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 890/1769] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:18:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:18:25.037836 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17922374-a207-4e63-9d30-499d81e4b8a4 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:25.039429 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-17922374-a207-4e63-9d30-499d81e4b8a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:18:25.058683 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17922374-a207-4e63-9d30-499d81e4b8a4 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:25.066917 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f2b1bc-2bd1-459d-b12f-f3061916049e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:25.067925 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f2b1bc-2bd1-459d-b12f-f3061916049e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:25.068277 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f2b1bc-2bd1-459d-b12f-f3061916049e admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:25.068619 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f2b1bc-2bd1-459d-b12f-f3061916049e admin admin] PATH_INFO: `/v3/projects/f835351847a8450c9344f052a327cf67` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:25.069999 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f2b1bc-2bd1-459d-b12f-f3061916049e admin admin] RBAC: Authorizing `identity:delete_project(project_id=f835351847a8450c9344f052a327cf67)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:25.072099 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f2b1bc-2bd1-459d-b12f-f3061916049e admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:25.124122 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-76f2b1bc-2bd1-459d-b12f-f3061916049e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f835351847a8450c9344f052a327cf67', 'request_id': 'req-76f2b1bc-2bd1-459d-b12f-f3061916049e'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:25.130945 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-76f2b1bc-2bd1-459d-b12f-f3061916049e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f835351847a8450c9344f052a327cf67', 'request_id': 'req-76f2b1bc-2bd1-459d-b12f-f3061916049e'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:25.136974 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-76f2b1bc-2bd1-459d-b12f-f3061916049e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f835351847a8450c9344f052a327cf67', 'request_id': 'req-76f2b1bc-2bd1-459d-b12f-f3061916049e'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:25.150937 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-76f2b1bc-2bd1-459d-b12f-f3061916049e admin admin] The token cache is being invalidate because project f835351847a8450c9344f052a327cf67 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80381) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 09:18:25.151309 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-76f2b1bc-2bd1-459d-b12f-f3061916049e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:25.152059 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-76f2b1bc-2bd1-459d-b12f-f3061916049e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:25.153641 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 880/1770] 10.222.0.26 () {58 vars in 1232 bytes} [Thu Jul 14 09:18:25 2022] DELETE /identity/v3/projects/f835351847a8450c9344f052a327cf67 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 09:18:25.165250 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e0232c-93f5-451c-8ed2-92b25828972a service neutron] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:25.175794 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e0232c-93f5-451c-8ed2-92b25828972a service neutron] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:25.196568 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:18:25.243850 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8156d7c0-92bb-4a04-a804-015b7498a48b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:25.245034 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8156d7c0-92bb-4a04-a804-015b7498a48b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:25.245467 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8156d7c0-92bb-4a04-a804-015b7498a48b admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:25.245834 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8156d7c0-92bb-4a04-a804-015b7498a48b admin admin] PATH_INFO: `/v3/users/03351ab734a143748d707a1b4650e90e` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:25.247231 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8156d7c0-92bb-4a04-a804-015b7498a48b admin admin] RBAC: Authorizing `identity:delete_user(user_id=03351ab734a143748d707a1b4650e90e)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:25.249449 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8156d7c0-92bb-4a04-a804-015b7498a48b admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:25.322151 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-8156d7c0-92bb-4a04-a804-015b7498a48b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '03351ab734a143748d707a1b4650e90e'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:25.323614 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-8156d7c0-92bb-4a04-a804-015b7498a48b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '03351ab734a143748d707a1b4650e90e'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:25.370293 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-8156d7c0-92bb-4a04-a804-015b7498a48b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '03351ab734a143748d707a1b4650e90e'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:25.396312 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-8156d7c0-92bb-4a04-a804-015b7498a48b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '03351ab734a143748d707a1b4650e90e'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:25.398139 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-8156d7c0-92bb-4a04-a804-015b7498a48b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '03351ab734a143748d707a1b4650e90e'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:25.435446 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 891/1771] 10.222.0.26 () {58 vars in 1223 bytes} [Thu Jul 14 09:18:25 2022] DELETE /identity/v3/users/03351ab734a143748d707a1b4650e90e => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:18:25.940214 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f2b1bc-2bd1-459d-b12f-f3061916049e admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:25.943388 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-76f2b1bc-2bd1-459d-b12f-f3061916049e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:18:25.966000 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8156d7c0-92bb-4a04-a804-015b7498a48b admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:25.966241 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f2b1bc-2bd1-459d-b12f-f3061916049e admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:25.967613 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-8156d7c0-92bb-4a04-a804-015b7498a48b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:18:25.989382 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:18:26.072801 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056e68e0-017e-4026-b92f-cb87a2225b26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:26.073936 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056e68e0-017e-4026-b92f-cb87a2225b26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:26.074412 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056e68e0-017e-4026-b92f-cb87a2225b26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:26.074886 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056e68e0-017e-4026-b92f-cb87a2225b26 admin admin] PATH_INFO: `/v3/projects/16d8a1d9b6254a9e9f44bda04b2880c3` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:26.075900 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8156d7c0-92bb-4a04-a804-015b7498a48b admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:26.076575 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056e68e0-017e-4026-b92f-cb87a2225b26 admin admin] RBAC: Authorizing `identity:delete_project(project_id=16d8a1d9b6254a9e9f44bda04b2880c3)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:26.079008 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056e68e0-017e-4026-b92f-cb87a2225b26 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:26.091759 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d17541-7c5b-4d93-bcb3-f9f35a4532b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:26.092750 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d17541-7c5b-4d93-bcb3-f9f35a4532b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:26.096152 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d17541-7c5b-4d93-bcb3-f9f35a4532b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:26.096370 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d17541-7c5b-4d93-bcb3-f9f35a4532b3 admin admin] PATH_INFO: `/v3/users/40699e94f2434942927d316e54ecb79a` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:26.100967 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d17541-7c5b-4d93-bcb3-f9f35a4532b3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=40699e94f2434942927d316e54ecb79a)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:26.104136 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d17541-7c5b-4d93-bcb3-f9f35a4532b3 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:26.177843 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-056e68e0-017e-4026-b92f-cb87a2225b26 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '16d8a1d9b6254a9e9f44bda04b2880c3', 'request_id': 'req-056e68e0-017e-4026-b92f-cb87a2225b26'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:26.185893 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-056e68e0-017e-4026-b92f-cb87a2225b26 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '16d8a1d9b6254a9e9f44bda04b2880c3', 'request_id': 'req-056e68e0-017e-4026-b92f-cb87a2225b26'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:26.196434 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-056e68e0-017e-4026-b92f-cb87a2225b26 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '16d8a1d9b6254a9e9f44bda04b2880c3', 'request_id': 'req-056e68e0-017e-4026-b92f-cb87a2225b26'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:26.197468 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:18:26.203224 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-35d17541-7c5b-4d93-bcb3-f9f35a4532b3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '40699e94f2434942927d316e54ecb79a'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:26.204369 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-35d17541-7c5b-4d93-bcb3-f9f35a4532b3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '40699e94f2434942927d316e54ecb79a'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:26.221788 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-056e68e0-017e-4026-b92f-cb87a2225b26 admin admin] The token cache is being invalidate because project 16d8a1d9b6254a9e9f44bda04b2880c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80381) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 09:18:26.222047 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-056e68e0-017e-4026-b92f-cb87a2225b26 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:26.222749 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-056e68e0-017e-4026-b92f-cb87a2225b26 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:26.225348 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 881/1772] 10.222.0.26 () {58 vars in 1232 bytes} [Thu Jul 14 09:18:25 2022] DELETE /identity/v3/projects/16d8a1d9b6254a9e9f44bda04b2880c3 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:18:26.234344 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-35d17541-7c5b-4d93-bcb3-f9f35a4532b3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '40699e94f2434942927d316e54ecb79a'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:26.236077 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ce6422-6c0d-41fa-9215-0a2b6772604c None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:26.237457 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ce6422-6c0d-41fa-9215-0a2b6772604c None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:26.237739 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ce6422-6c0d-41fa-9215-0a2b6772604c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:26.249412 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-35d17541-7c5b-4d93-bcb3-f9f35a4532b3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '40699e94f2434942927d316e54ecb79a'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:26.250790 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-35d17541-7c5b-4d93-bcb3-f9f35a4532b3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '40699e94f2434942927d316e54ecb79a'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:26.260642 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-b9ce6422-6c0d-41fa-9215-0a2b6772604c None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:18:26.262449 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-b9ce6422-6c0d-41fa-9215-0a2b6772604c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:18:26.280117 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 892/1773] 10.222.0.26 () {58 vars in 1223 bytes} [Thu Jul 14 09:18:25 2022] DELETE /identity/v3/users/40699e94f2434942927d316e54ecb79a => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 09:18:26.281895 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e728a8-410a-4c04-8881-a82d97ffaa24 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:26.281895 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e728a8-410a-4c04-8881-a82d97ffaa24 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:26.282280 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e728a8-410a-4c04-8881-a82d97ffaa24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:26.308331 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-e2e728a8-410a-4c04-8881-a82d97ffaa24 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:18:26.310256 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-e2e728a8-410a-4c04-8881-a82d97ffaa24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:18:26.390578 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 893/1774] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:18:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:18:26.398011 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e728a8-410a-4c04-8881-a82d97ffaa24 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:26.400795 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-e2e728a8-410a-4c04-8881-a82d97ffaa24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:18:26.424357 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 882/1775] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:18:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:18:26.429448 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c437cab-1b14-44ca-bcf9-f3e4886e0002 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:26.429879 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c437cab-1b14-44ca-bcf9-f3e4886e0002 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:26.430272 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c437cab-1b14-44ca-bcf9-f3e4886e0002 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:26.453383 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-1c437cab-1b14-44ca-bcf9-f3e4886e0002 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:18:26.455101 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-1c437cab-1b14-44ca-bcf9-f3e4886e0002 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:18:26.464576 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 883/1776] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:18:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 09:18:26.468824 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c437cab-1b14-44ca-bcf9-f3e4886e0002 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:26.479713 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c437cab-1b14-44ca-bcf9-f3e4886e0002 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:26.482091 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e728a8-410a-4c04-8881-a82d97ffaa24 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:26.485319 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59f71b7-cad2-417c-a50d-218158a16872 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:26.486035 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59f71b7-cad2-417c-a50d-218158a16872 admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:26.486344 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59f71b7-cad2-417c-a50d-218158a16872 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:26.486604 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59f71b7-cad2-417c-a50d-218158a16872 admin admin] PATH_INFO: `/v3/domains` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:26.487434 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59f71b7-cad2-417c-a50d-218158a16872 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:18:26.487751 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59f71b7-cad2-417c-a50d-218158a16872 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:26.489452 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59f71b7-cad2-417c-a50d-218158a16872 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:26.489960 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecb95f4-0d70-438f-b02b-c0ac804d90a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '952911c492444ffea4b211a24b564117', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:26.490668 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecb95f4-0d70-438f-b02b-c0ac804d90a8 service neutron] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:26.490957 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecb95f4-0d70-438f-b02b-c0ac804d90a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:26.493045 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecb95f4-0d70-438f-b02b-c0ac804d90a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:26.504143 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 884/1777] 10.222.0.26 () {58 vars in 1152 bytes} [Thu Jul 14 09:18:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:18:26.508072 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecb95f4-0d70-438f-b02b-c0ac804d90a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:26.508471 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59f71b7-cad2-417c-a50d-218158a16872 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:26.509572 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecb95f4-0d70-438f-b02b-c0ac804d90a8 service neutron] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:26.520441 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59f71b7-cad2-417c-a50d-218158a16872 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:26.521948 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 894/1778] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:18:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:18:26.527407 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e92177-a8da-4738-8e9c-5056832f2983 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:26.528258 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e92177-a8da-4738-8e9c-5056832f2983 admin admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:26.528550 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e92177-a8da-4738-8e9c-5056832f2983 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:26.528831 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e92177-a8da-4738-8e9c-5056832f2983 admin admin] PATH_INFO: `/v3/projects` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:26.529689 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e92177-a8da-4738-8e9c-5056832f2983 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:26.531715 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e92177-a8da-4738-8e9c-5056832f2983 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:26.581899 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 885/1779] 10.222.0.26 () {60 vars in 1152 bytes} [Thu Jul 14 09:18:26 2022] POST /identity/v3/projects => generated 381 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 09:18:26.587074 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecb95f4-0d70-438f-b02b-c0ac804d90a8 service neutron] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:26.604667 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecb95f4-0d70-438f-b02b-c0ac804d90a8 service neutron] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:26.742159 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236567d3-0ac5-4194-94b7-6dd8034c8e62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:26.743799 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236567d3-0ac5-4194-94b7-6dd8034c8e62 admin admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:26.744385 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236567d3-0ac5-4194-94b7-6dd8034c8e62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:26.744385 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236567d3-0ac5-4194-94b7-6dd8034c8e62 admin admin] PATH_INFO: `/v3/users` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:26.747839 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236567d3-0ac5-4194-94b7-6dd8034c8e62 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:26.766167 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236567d3-0ac5-4194-94b7-6dd8034c8e62 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:26.790505 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:DELETE=4,keystone:SELECT=93,keystone:INSERT=3 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:18:26.819424 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 895/1780] 10.222.0.26 () {60 vars in 1143 bytes} [Thu Jul 14 09:18:26 2022] POST /identity/v3/users => generated 291 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 09:18:26.821249 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e92177-a8da-4738-8e9c-5056832f2983 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:26.837717 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e92177-a8da-4738-8e9c-5056832f2983 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:26.853291 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58de7cd5-bd14-4cf5-8146-a6c00449ee64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:26.854099 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58de7cd5-bd14-4cf5-8146-a6c00449ee64 admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:26.854439 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58de7cd5-bd14-4cf5-8146-a6c00449ee64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:26.854804 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58de7cd5-bd14-4cf5-8146-a6c00449ee64 admin admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:26.855704 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58de7cd5-bd14-4cf5-8146-a6c00449ee64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:26.857407 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58de7cd5-bd14-4cf5-8146-a6c00449ee64 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:26.865796 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:18:26.882967 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 886/1781] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jul 14 09:18:26 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 09:18:26.885426 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236567d3-0ac5-4194-94b7-6dd8034c8e62 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:26.903177 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236567d3-0ac5-4194-94b7-6dd8034c8e62 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:26.913419 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21ca534-de3e-433a-99d7-821ea81af43b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:26.914353 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21ca534-de3e-433a-99d7-821ea81af43b admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:26.914629 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21ca534-de3e-433a-99d7-821ea81af43b admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:26.914906 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21ca534-de3e-433a-99d7-821ea81af43b admin admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:26.916151 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21ca534-de3e-433a-99d7-821ea81af43b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:26.917951 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21ca534-de3e-433a-99d7-821ea81af43b admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:26.938728 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 896/1782] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jul 14 09:18:26 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 09:18:26.939243 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58de7cd5-bd14-4cf5-8146-a6c00449ee64 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:26.956568 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58de7cd5-bd14-4cf5-8146-a6c00449ee64 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:26.968676 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f550074e-99bd-429c-a3f0-e2f6bc88c09f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:26.970775 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f550074e-99bd-429c-a3f0-e2f6bc88c09f admin admin] REQUEST_METHOD: `PUT` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:26.973448 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f550074e-99bd-429c-a3f0-e2f6bc88c09f admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:26.973918 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f550074e-99bd-429c-a3f0-e2f6bc88c09f admin admin] PATH_INFO: `/v3/projects/ddf9a1f821894281bd344b0f28c1dcc4/users/8c7da06d0a334c458fba37e5edbb0c52/roles/d1691322ce6a4edab7ce722edf0f2988` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:26.998041 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f550074e-99bd-429c-a3f0-e2f6bc88c09f admin admin] RBAC: Authorizing `identity:create_grant(project_id=ddf9a1f821894281bd344b0f28c1dcc4, user_id=8c7da06d0a334c458fba37e5edbb0c52, role_id=d1691322ce6a4edab7ce722edf0f2988)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:27.001316 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f550074e-99bd-429c-a3f0-e2f6bc88c09f admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:27.020763 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 887/1783] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jul 14 09:18:26 2022] PUT /identity/v3/projects/ddf9a1f821894281bd344b0f28c1dcc4/users/8c7da06d0a334c458fba37e5edbb0c52/roles/d1691322ce6a4edab7ce722edf0f2988 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:18:27.033632 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6aa7b2-2016-4336-8c2f-fe22c03a0e77 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:27.034008 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6aa7b2-2016-4336-8c2f-fe22c03a0e77 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:27.034445 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6aa7b2-2016-4336-8c2f-fe22c03a0e77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:27.048798 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f550074e-99bd-429c-a3f0-e2f6bc88c09f admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:27.050581 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-f550074e-99bd-429c-a3f0-e2f6bc88c09f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:18:27.110788 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-9c6aa7b2-2016-4336-8c2f-fe22c03a0e77 None None] MFA Rules not processed for user `8c7da06d0a334c458fba37e5edbb0c52`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:18:27.112328 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-9c6aa7b2-2016-4336-8c2f-fe22c03a0e77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:18:27.190749 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f550074e-99bd-429c-a3f0-e2f6bc88c09f admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:27.201483 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26c762a-b22f-4da0-98de-827d21f0d99f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:27.202429 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26c762a-b22f-4da0-98de-827d21f0d99f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:27.202654 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26c762a-b22f-4da0-98de-827d21f0d99f admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:27.203406 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26c762a-b22f-4da0-98de-827d21f0d99f admin admin] PATH_INFO: `/v3/projects/43cf74a97d964c4f98c182a37573ebb3` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:27.204305 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26c762a-b22f-4da0-98de-827d21f0d99f admin admin] RBAC: Authorizing `identity:delete_project(project_id=43cf74a97d964c4f98c182a37573ebb3)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:27.206249 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26c762a-b22f-4da0-98de-827d21f0d99f admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:27.277007 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 897/1784] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:18:27 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:18:27.281600 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6aa7b2-2016-4336-8c2f-fe22c03a0e77 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:27.288836 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-d26c762a-b22f-4da0-98de-827d21f0d99f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '43cf74a97d964c4f98c182a37573ebb3', 'request_id': 'req-d26c762a-b22f-4da0-98de-827d21f0d99f'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:27.297167 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-d26c762a-b22f-4da0-98de-827d21f0d99f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '43cf74a97d964c4f98c182a37573ebb3', 'request_id': 'req-d26c762a-b22f-4da0-98de-827d21f0d99f'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:27.298473 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6aa7b2-2016-4336-8c2f-fe22c03a0e77 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:27.303524 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-d26c762a-b22f-4da0-98de-827d21f0d99f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '43cf74a97d964c4f98c182a37573ebb3', 'request_id': 'req-d26c762a-b22f-4da0-98de-827d21f0d99f'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:27.322192 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-d26c762a-b22f-4da0-98de-827d21f0d99f admin admin] The token cache is being invalidate because project 43cf74a97d964c4f98c182a37573ebb3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80381) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 09:18:27.322770 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-d26c762a-b22f-4da0-98de-827d21f0d99f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:27.323982 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-d26c762a-b22f-4da0-98de-827d21f0d99f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:27.341414 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 888/1785] 10.222.0.26 () {58 vars in 1232 bytes} [Thu Jul 14 09:18:27 2022] DELETE /identity/v3/projects/43cf74a97d964c4f98c182a37573ebb3 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 09:18:27.341414 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952a652f-820d-4d99-9dbd-c485a0962c84 None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:27.341414 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952a652f-820d-4d99-9dbd-c485a0962c84 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:27.344112 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952a652f-820d-4d99-9dbd-c485a0962c84 None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:27.345623 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 889/1786] 10.222.0.26 () {54 vars in 921 bytes} [Thu Jul 14 09:18:27 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:18:27.346361 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952a652f-820d-4d99-9dbd-c485a0962c84 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:27.350896 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-952a652f-820d-4d99-9dbd-c485a0962c84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:18:27.381226 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59457d21-1464-4940-8673-4e7d7992610b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:27.383330 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59457d21-1464-4940-8673-4e7d7992610b admin admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:27.383830 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59457d21-1464-4940-8673-4e7d7992610b admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:27.384476 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59457d21-1464-4940-8673-4e7d7992610b admin admin] PATH_INFO: `/v3/projects` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:27.385431 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59457d21-1464-4940-8673-4e7d7992610b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:27.388106 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59457d21-1464-4940-8673-4e7d7992610b admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:27.440000 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 898/1787] 10.222.0.26 () {60 vars in 1152 bytes} [Thu Jul 14 09:18:27 2022] POST /identity/v3/projects => generated 381 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 09:18:27.441942 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0ff942-2cc9-4b4c-a335-526ac07ed003 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:27.442422 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0ff942-2cc9-4b4c-a335-526ac07ed003 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:27.442899 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0ff942-2cc9-4b4c-a335-526ac07ed003 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:27.449744 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952a652f-820d-4d99-9dbd-c485a0962c84 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:27.470473 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:18:27.481325 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-fb0ff942-2cc9-4b4c-a335-526ac07ed003 None None] MFA Rules not processed for user `538fff3c5a2b487eb44ae7c1a1198102`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:18:27.482585 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-fb0ff942-2cc9-4b4c-a335-526ac07ed003 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:18:27.571750 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ff3fe4-8c65-4b2d-9df3-c211b1f7e6f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:27.572807 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ff3fe4-8c65-4b2d-9df3-c211b1f7e6f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:27.573093 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ff3fe4-8c65-4b2d-9df3-c211b1f7e6f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:27.573381 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ff3fe4-8c65-4b2d-9df3-c211b1f7e6f1 admin admin] PATH_INFO: `/v3/users/7603d171e1f7402db55ea79b247739b5` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:27.574997 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ff3fe4-8c65-4b2d-9df3-c211b1f7e6f1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7603d171e1f7402db55ea79b247739b5)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:27.577305 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ff3fe4-8c65-4b2d-9df3-c211b1f7e6f1 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:27.577917 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:18:27.608296 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 899/1788] 10.222.0.26 () {58 vars in 1018 bytes} [Thu Jul 14 09:18:27 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:18:27.609813 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0ff942-2cc9-4b4c-a335-526ac07ed003 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:27.612305 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-fb0ff942-2cc9-4b4c-a335-526ac07ed003 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:18:27.627144 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0ff942-2cc9-4b4c-a335-526ac07ed003 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:27.632527 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e952ffa7-3618-44c1-9e38-e2979f4c5126 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:27.632832 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e952ffa7-3618-44c1-9e38-e2979f4c5126 admin admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:27.633120 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e952ffa7-3618-44c1-9e38-e2979f4c5126 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:27.633398 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e952ffa7-3618-44c1-9e38-e2979f4c5126 admin admin] PATH_INFO: `/v3/users` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:27.634094 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e952ffa7-3618-44c1-9e38-e2979f4c5126 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:27.636123 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e952ffa7-3618-44c1-9e38-e2979f4c5126 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:27.658246 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-c3ff3fe4-8c65-4b2d-9df3-c211b1f7e6f1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7603d171e1f7402db55ea79b247739b5'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:27.663568 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 900/1789] 10.222.0.26 () {60 vars in 1143 bytes} [Thu Jul 14 09:18:27 2022] POST /identity/v3/users => generated 297 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 09:18:27.664415 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e952ffa7-3618-44c1-9e38-e2979f4c5126 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:27.669641 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-e952ffa7-3618-44c1-9e38-e2979f4c5126 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:18:27.672463 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-c3ff3fe4-8c65-4b2d-9df3-c211b1f7e6f1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7603d171e1f7402db55ea79b247739b5'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:27.675584 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-c3ff3fe4-8c65-4b2d-9df3-c211b1f7e6f1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7603d171e1f7402db55ea79b247739b5'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:27.677384 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-c3ff3fe4-8c65-4b2d-9df3-c211b1f7e6f1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7603d171e1f7402db55ea79b247739b5'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:27.692663 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-c3ff3fe4-8c65-4b2d-9df3-c211b1f7e6f1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7603d171e1f7402db55ea79b247739b5'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:27.694667 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e952ffa7-3618-44c1-9e38-e2979f4c5126 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:27.702404 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c029ece1-a13f-4621-b4ea-4c55a73228e8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '538fff3c5a2b487eb44ae7c1a1198102', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:27.703465 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c029ece1-a13f-4621-b4ea-4c55a73228e8 service nova] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:27.703465 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c029ece1-a13f-4621-b4ea-4c55a73228e8 service nova] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:27.703787 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c029ece1-a13f-4621-b4ea-4c55a73228e8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:27.705916 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-c029ece1-a13f-4621-b4ea-4c55a73228e8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:18:27.721323 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c029ece1-a13f-4621-b4ea-4c55a73228e8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:27.723478 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c029ece1-a13f-4621-b4ea-4c55a73228e8 service nova] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:27.735736 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 890/1790] 10.222.0.26 () {58 vars in 1223 bytes} [Thu Jul 14 09:18:27 2022] DELETE /identity/v3/users/7603d171e1f7402db55ea79b247739b5 => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:18:27.735736 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ff3fe4-8c65-4b2d-9df3-c211b1f7e6f1 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:27.735736 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-c3ff3fe4-8c65-4b2d-9df3-c211b1f7e6f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:18:27.754064 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 901/1791] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:18:27 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:18:27.879162 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ff3fe4-8c65-4b2d-9df3-c211b1f7e6f1 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:27.890078 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1b88c9-9955-4bcc-8eaa-c43d00bdc36b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:27.890506 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1b88c9-9955-4bcc-8eaa-c43d00bdc36b admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:27.890738 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1b88c9-9955-4bcc-8eaa-c43d00bdc36b admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:27.891161 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1b88c9-9955-4bcc-8eaa-c43d00bdc36b admin admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:27.891939 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1b88c9-9955-4bcc-8eaa-c43d00bdc36b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:27.893546 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1b88c9-9955-4bcc-8eaa-c43d00bdc36b admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:27.906080 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 891/1792] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jul 14 09:18:27 2022] GET /identity/v3/roles => generated 1420 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 09:18:27.910232 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c029ece1-a13f-4621-b4ea-4c55a73228e8 service nova] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:27.925964 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c029ece1-a13f-4621-b4ea-4c55a73228e8 service nova] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:27.936410 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b72205-b7d3-4592-a51c-d4609a1e23b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:27.939253 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b72205-b7d3-4592-a51c-d4609a1e23b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:27.939253 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b72205-b7d3-4592-a51c-d4609a1e23b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:27.939253 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b72205-b7d3-4592-a51c-d4609a1e23b7 admin admin] PATH_INFO: `/v3/projects/714e720078134fbea6740d5c0c7a4bcb/users/f1907a1379e448108088c3e93e613749/roles/7b549d58d1724395b343609d895c80d0` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:27.964080 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b72205-b7d3-4592-a51c-d4609a1e23b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=714e720078134fbea6740d5c0c7a4bcb, user_id=f1907a1379e448108088c3e93e613749, role_id=7b549d58d1724395b343609d895c80d0)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:27.967020 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b72205-b7d3-4592-a51c-d4609a1e23b7 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:27.989810 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 902/1793] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jul 14 09:18:27 2022] PUT /identity/v3/projects/714e720078134fbea6740d5c0c7a4bcb/users/f1907a1379e448108088c3e93e613749/roles/7b549d58d1724395b343609d895c80d0 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:18:27.993601 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1b88c9-9955-4bcc-8eaa-c43d00bdc36b admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:28.017446 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1b88c9-9955-4bcc-8eaa-c43d00bdc36b admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:28.054102 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:18:28.142053 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b72205-b7d3-4592-a51c-d4609a1e23b7 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:28.153379 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-05b72205-b7d3-4592-a51c-d4609a1e23b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:18:28.160408 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db71a10e-590c-4be5-9479-edf20be4a338 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:28.162352 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db71a10e-590c-4be5-9479-edf20be4a338 admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:28.163767 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db71a10e-590c-4be5-9479-edf20be4a338 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:28.164037 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db71a10e-590c-4be5-9479-edf20be4a338 admin admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:28.164911 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db71a10e-590c-4be5-9479-edf20be4a338 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:28.166947 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db71a10e-590c-4be5-9479-edf20be4a338 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:28.175072 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b72205-b7d3-4592-a51c-d4609a1e23b7 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:28.192940 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db749095-ffe7-4bbe-aaf7-86f35f52f219 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:28.193894 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db749095-ffe7-4bbe-aaf7-86f35f52f219 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:28.194174 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db749095-ffe7-4bbe-aaf7-86f35f52f219 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:28.194455 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db749095-ffe7-4bbe-aaf7-86f35f52f219 admin admin] PATH_INFO: `/v3/projects/9ce4daf557254d97b4b3855f645c4ae9` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:28.195689 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db749095-ffe7-4bbe-aaf7-86f35f52f219 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9ce4daf557254d97b4b3855f645c4ae9)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:28.198275 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db749095-ffe7-4bbe-aaf7-86f35f52f219 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:28.199177 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 892/1794] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jul 14 09:18:27 2022] GET /identity/v3/roles => generated 1420 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 09:18:28.201015 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db71a10e-590c-4be5-9479-edf20be4a338 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:28.228261 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db71a10e-590c-4be5-9479-edf20be4a338 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:28.241450 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e39cf4-5615-426d-8468-2a070b47ba1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:28.245817 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e39cf4-5615-426d-8468-2a070b47ba1c admin admin] REQUEST_METHOD: `PUT` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:28.245917 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e39cf4-5615-426d-8468-2a070b47ba1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:28.248779 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e39cf4-5615-426d-8468-2a070b47ba1c admin admin] PATH_INFO: `/v3/projects/714e720078134fbea6740d5c0c7a4bcb/users/f1907a1379e448108088c3e93e613749/roles/7b549d58d1724395b343609d895c80d0` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:28.253877 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e39cf4-5615-426d-8468-2a070b47ba1c admin admin] RBAC: Authorizing `identity:create_grant(project_id=714e720078134fbea6740d5c0c7a4bcb, user_id=f1907a1379e448108088c3e93e613749, role_id=7b549d58d1724395b343609d895c80d0)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:28.260428 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e39cf4-5615-426d-8468-2a070b47ba1c admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:28.294379 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-db749095-ffe7-4bbe-aaf7-86f35f52f219 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9ce4daf557254d97b4b3855f645c4ae9', 'request_id': 'req-db749095-ffe7-4bbe-aaf7-86f35f52f219'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:28.295548 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 893/1795] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jul 14 09:18:28 2022] PUT /identity/v3/projects/714e720078134fbea6740d5c0c7a4bcb/users/f1907a1379e448108088c3e93e613749/roles/7b549d58d1724395b343609d895c80d0 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:18:28.306491 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-db749095-ffe7-4bbe-aaf7-86f35f52f219 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9ce4daf557254d97b4b3855f645c4ae9', 'request_id': 'req-db749095-ffe7-4bbe-aaf7-86f35f52f219'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:28.308155 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38755214-358b-4204-b43e-d7806764a712 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:28.308423 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38755214-358b-4204-b43e-d7806764a712 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:28.308810 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38755214-358b-4204-b43e-d7806764a712 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:28.325165 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-db749095-ffe7-4bbe-aaf7-86f35f52f219 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9ce4daf557254d97b4b3855f645c4ae9', 'request_id': 'req-db749095-ffe7-4bbe-aaf7-86f35f52f219'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:28.354242 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-db749095-ffe7-4bbe-aaf7-86f35f52f219 admin admin] The token cache is being invalidate because project 9ce4daf557254d97b4b3855f645c4ae9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80382) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 09:18:28.355930 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-db749095-ffe7-4bbe-aaf7-86f35f52f219 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:28.358736 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-db749095-ffe7-4bbe-aaf7-86f35f52f219 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:28.369825 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 903/1796] 10.222.0.26 () {58 vars in 1232 bytes} [Thu Jul 14 09:18:28 2022] DELETE /identity/v3/projects/9ce4daf557254d97b4b3855f645c4ae9 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:18:28.370924 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db749095-ffe7-4bbe-aaf7-86f35f52f219 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:28.374096 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-db749095-ffe7-4bbe-aaf7-86f35f52f219 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:18:28.378448 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-38755214-358b-4204-b43e-d7806764a712 None None] MFA Rules not processed for user `f1907a1379e448108088c3e93e613749`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:18:28.382683 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-38755214-358b-4204-b43e-d7806764a712 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:18:28.522393 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db749095-ffe7-4bbe-aaf7-86f35f52f219 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:28.555580 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:18:28.579127 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 894/1797] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:18:28 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 09:18:28.593837 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38755214-358b-4204-b43e-d7806764a712 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:28.596008 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-38755214-358b-4204-b43e-d7806764a712 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:18:28.631696 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59355d23-0562-4955-b7fe-95f2080a37b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:28.633280 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59355d23-0562-4955-b7fe-95f2080a37b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:28.633872 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59355d23-0562-4955-b7fe-95f2080a37b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:28.634297 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59355d23-0562-4955-b7fe-95f2080a37b7 admin admin] PATH_INFO: `/v3/users/beca41779b324b2782d94d73cb8bba1f` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:28.636170 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59355d23-0562-4955-b7fe-95f2080a37b7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=beca41779b324b2782d94d73cb8bba1f)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:28.639062 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59355d23-0562-4955-b7fe-95f2080a37b7 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:28.753363 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38755214-358b-4204-b43e-d7806764a712 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:28.756833 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-59355d23-0562-4955-b7fe-95f2080a37b7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'beca41779b324b2782d94d73cb8bba1f'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:28.758653 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-59355d23-0562-4955-b7fe-95f2080a37b7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'beca41779b324b2782d94d73cb8bba1f'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:28.763963 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6d4acc-5802-4943-bcf8-13bda0e456db service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e76f1952a5b146808844b9f2b7395035', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:28.768887 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6d4acc-5802-4943-bcf8-13bda0e456db service cinder] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:28.769491 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6d4acc-5802-4943-bcf8-13bda0e456db service cinder] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:28.769800 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6d4acc-5802-4943-bcf8-13bda0e456db service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:28.782613 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-59355d23-0562-4955-b7fe-95f2080a37b7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'beca41779b324b2782d94d73cb8bba1f'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:28.786990 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-cb6d4acc-5802-4943-bcf8-13bda0e456db service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:18:28.810102 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-59355d23-0562-4955-b7fe-95f2080a37b7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'beca41779b324b2782d94d73cb8bba1f'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:28.811698 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-59355d23-0562-4955-b7fe-95f2080a37b7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'beca41779b324b2782d94d73cb8bba1f'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:28.817826 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6d4acc-5802-4943-bcf8-13bda0e456db service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:28.821589 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6d4acc-5802-4943-bcf8-13bda0e456db service cinder] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:28.838365 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 904/1798] 10.222.0.26 () {58 vars in 1223 bytes} [Thu Jul 14 09:18:28 2022] DELETE /identity/v3/users/beca41779b324b2782d94d73cb8bba1f => generated 0 bytes in 468 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:18:28.841258 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 895/1799] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:18:28 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 247 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 09:18:29.236334 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59355d23-0562-4955-b7fe-95f2080a37b7 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:29.237840 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-59355d23-0562-4955-b7fe-95f2080a37b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:18:29.346213 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59355d23-0562-4955-b7fe-95f2080a37b7 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:29.351860 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1602f7ca-0925-4395-b2f7-4a70f288e8fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:29.352947 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1602f7ca-0925-4395-b2f7-4a70f288e8fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:29.353238 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1602f7ca-0925-4395-b2f7-4a70f288e8fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:29.353609 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1602f7ca-0925-4395-b2f7-4a70f288e8fc admin admin] PATH_INFO: `/v3/projects/4fab3c6360fd4339a534f0ed1fb80cec` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:29.356143 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1602f7ca-0925-4395-b2f7-4a70f288e8fc admin admin] RBAC: Authorizing `identity:delete_project(project_id=4fab3c6360fd4339a534f0ed1fb80cec)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:29.360234 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1602f7ca-0925-4395-b2f7-4a70f288e8fc admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:29.403673 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6d4acc-5802-4943-bcf8-13bda0e456db service cinder] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:29.419764 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6d4acc-5802-4943-bcf8-13bda0e456db service cinder] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:29.458407 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:18:29.464601 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-1602f7ca-0925-4395-b2f7-4a70f288e8fc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4fab3c6360fd4339a534f0ed1fb80cec', 'request_id': 'req-1602f7ca-0925-4395-b2f7-4a70f288e8fc'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:29.477049 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-1602f7ca-0925-4395-b2f7-4a70f288e8fc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4fab3c6360fd4339a534f0ed1fb80cec', 'request_id': 'req-1602f7ca-0925-4395-b2f7-4a70f288e8fc'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:29.490060 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-1602f7ca-0925-4395-b2f7-4a70f288e8fc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4fab3c6360fd4339a534f0ed1fb80cec', 'request_id': 'req-1602f7ca-0925-4395-b2f7-4a70f288e8fc'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:29.492462 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:18:29.507955 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-1602f7ca-0925-4395-b2f7-4a70f288e8fc admin admin] The token cache is being invalidate because project 4fab3c6360fd4339a534f0ed1fb80cec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80382) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 09:18:29.508248 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-1602f7ca-0925-4395-b2f7-4a70f288e8fc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:29.508897 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-1602f7ca-0925-4395-b2f7-4a70f288e8fc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:29.511000 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 905/1800] 10.222.0.26 () {58 vars in 1232 bytes} [Thu Jul 14 09:18:29 2022] DELETE /identity/v3/projects/4fab3c6360fd4339a534f0ed1fb80cec => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:18:29.514424 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54902d6-d123-4724-ada7-f882db2c7956 tempest-VolumeQuotaClassesTest-1837463828 tempest-VolumeQuotaClassesTest-1837463828-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1907a1379e448108088c3e93e613749', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '714e720078134fbea6740d5c0c7a4bcb', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:29.515495 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54902d6-d123-4724-ada7-f882db2c7956 tempest-VolumeQuotaClassesTest-1837463828 tempest-VolumeQuotaClassesTest-1837463828-project-admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:29.515792 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54902d6-d123-4724-ada7-f882db2c7956 tempest-VolumeQuotaClassesTest-1837463828 tempest-VolumeQuotaClassesTest-1837463828-project-admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:29.516087 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54902d6-d123-4724-ada7-f882db2c7956 tempest-VolumeQuotaClassesTest-1837463828 tempest-VolumeQuotaClassesTest-1837463828-project-admin] PATH_INFO: `/v3/domains` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:29.516822 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e64c29-5a8b-4191-a90b-d1623b15b1b2 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:29.516939 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54902d6-d123-4724-ada7-f882db2c7956 tempest-VolumeQuotaClassesTest-1837463828 tempest-VolumeQuotaClassesTest-1837463828-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:18:29.517109 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e64c29-5a8b-4191-a90b-d1623b15b1b2 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:29.517200 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54902d6-d123-4724-ada7-f882db2c7956 tempest-VolumeQuotaClassesTest-1837463828 tempest-VolumeQuotaClassesTest-1837463828-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:29.517436 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e64c29-5a8b-4191-a90b-d1623b15b1b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:29.518743 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54902d6-d123-4724-ada7-f882db2c7956 tempest-VolumeQuotaClassesTest-1837463828 tempest-VolumeQuotaClassesTest-1837463828-project-admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:29.535355 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 896/1801] 10.222.0.26 () {58 vars in 1152 bytes} [Thu Jul 14 09:18:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 09:18:29.537970 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54902d6-d123-4724-ada7-f882db2c7956 tempest-VolumeQuotaClassesTest-1837463828 tempest-VolumeQuotaClassesTest-1837463828-project-admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:29.539685 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-c54902d6-d123-4724-ada7-f882db2c7956 tempest-VolumeQuotaClassesTest-1837463828 tempest-VolumeQuotaClassesTest-1837463828-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:18:29.540795 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-43e64c29-5a8b-4191-a90b-d1623b15b1b2 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:18:29.542045 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-43e64c29-5a8b-4191-a90b-d1623b15b1b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:18:29.558123 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54902d6-d123-4724-ada7-f882db2c7956 tempest-VolumeQuotaClassesTest-1837463828 tempest-VolumeQuotaClassesTest-1837463828-project-admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:29.609335 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 906/1802] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:18:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:18:29.618544 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c468771f-b109-4d68-bce0-5f9141565caa None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:29.618806 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c468771f-b109-4d68-bce0-5f9141565caa None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:29.619077 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c468771f-b109-4d68-bce0-5f9141565caa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:29.629835 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86401d8-7126-4c1c-a029-f7e8c30db6a7 tempest-VolumeQuotaClassesTest-1837463828 tempest-VolumeQuotaClassesTest-1837463828-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1907a1379e448108088c3e93e613749', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '714e720078134fbea6740d5c0c7a4bcb', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:29.630792 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86401d8-7126-4c1c-a029-f7e8c30db6a7 tempest-VolumeQuotaClassesTest-1837463828 tempest-VolumeQuotaClassesTest-1837463828-project-admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:29.631293 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86401d8-7126-4c1c-a029-f7e8c30db6a7 tempest-VolumeQuotaClassesTest-1837463828 tempest-VolumeQuotaClassesTest-1837463828-project-admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:29.631729 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86401d8-7126-4c1c-a029-f7e8c30db6a7 tempest-VolumeQuotaClassesTest-1837463828 tempest-VolumeQuotaClassesTest-1837463828-project-admin] PATH_INFO: `/v3/projects` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:29.633324 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86401d8-7126-4c1c-a029-f7e8c30db6a7 tempest-VolumeQuotaClassesTest-1837463828 tempest-VolumeQuotaClassesTest-1837463828-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:29.635887 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86401d8-7126-4c1c-a029-f7e8c30db6a7 tempest-VolumeQuotaClassesTest-1837463828 tempest-VolumeQuotaClassesTest-1837463828-project-admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:29.638224 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-c468771f-b109-4d68-bce0-5f9141565caa None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:18:29.639551 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-c468771f-b109-4d68-bce0-5f9141565caa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:18:29.654507 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 907/1803] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:18:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 09:18:29.660458 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c468771f-b109-4d68-bce0-5f9141565caa None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:29.676116 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c468771f-b109-4d68-bce0-5f9141565caa None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:29.692907 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 897/1804] 10.222.0.26 () {60 vars in 1152 bytes} [Thu Jul 14 09:18:29 2022] POST /identity/v3/projects => generated 353 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 09:18:29.693133 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6caea002-81b0-49e8-b12c-9baf68b28117 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:29.693133 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caea002-81b0-49e8-b12c-9baf68b28117 admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:29.693133 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caea002-81b0-49e8-b12c-9baf68b28117 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:29.693414 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caea002-81b0-49e8-b12c-9baf68b28117 admin admin] PATH_INFO: `/v3/domains` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:29.694038 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6caea002-81b0-49e8-b12c-9baf68b28117 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:18:29.694352 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6caea002-81b0-49e8-b12c-9baf68b28117 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:29.695091 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86401d8-7126-4c1c-a029-f7e8c30db6a7 tempest-VolumeQuotaClassesTest-1837463828 tempest-VolumeQuotaClassesTest-1837463828-project-admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:29.696011 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6caea002-81b0-49e8-b12c-9baf68b28117 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:29.716710 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 908/1805] 10.222.0.26 () {58 vars in 1152 bytes} [Thu Jul 14 09:18:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:18:29.720561 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6caea002-81b0-49e8-b12c-9baf68b28117 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:29.739972 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6caea002-81b0-49e8-b12c-9baf68b28117 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:29.798426 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86401d8-7126-4c1c-a029-f7e8c30db6a7 tempest-VolumeQuotaClassesTest-1837463828 tempest-VolumeQuotaClassesTest-1837463828-project-admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:29.804814 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4ce5ea-ff8d-4948-96c7-10d6f6783f15 tempest-VolumeQuotaClassesTest-1837463828 tempest-VolumeQuotaClassesTest-1837463828-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1907a1379e448108088c3e93e613749', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '714e720078134fbea6740d5c0c7a4bcb', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:29.806096 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4ce5ea-ff8d-4948-96c7-10d6f6783f15 tempest-VolumeQuotaClassesTest-1837463828 tempest-VolumeQuotaClassesTest-1837463828-project-admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:29.806096 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4ce5ea-ff8d-4948-96c7-10d6f6783f15 tempest-VolumeQuotaClassesTest-1837463828 tempest-VolumeQuotaClassesTest-1837463828-project-admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:29.806313 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4ce5ea-ff8d-4948-96c7-10d6f6783f15 tempest-VolumeQuotaClassesTest-1837463828 tempest-VolumeQuotaClassesTest-1837463828-project-admin] PATH_INFO: `/v3/domains` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:29.806888 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4ce5ea-ff8d-4948-96c7-10d6f6783f15 tempest-VolumeQuotaClassesTest-1837463828 tempest-VolumeQuotaClassesTest-1837463828-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:18:29.807234 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4ce5ea-ff8d-4948-96c7-10d6f6783f15 tempest-VolumeQuotaClassesTest-1837463828 tempest-VolumeQuotaClassesTest-1837463828-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:29.808149 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c65103-b571-439b-b6ce-ba484503aa5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:29.810430 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4ce5ea-ff8d-4948-96c7-10d6f6783f15 tempest-VolumeQuotaClassesTest-1837463828 tempest-VolumeQuotaClassesTest-1837463828-project-admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:29.810608 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c65103-b571-439b-b6ce-ba484503aa5f admin admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:29.810608 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c65103-b571-439b-b6ce-ba484503aa5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:29.810608 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c65103-b571-439b-b6ce-ba484503aa5f admin admin] PATH_INFO: `/v3/projects` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:29.810608 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c65103-b571-439b-b6ce-ba484503aa5f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:29.812093 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c65103-b571-439b-b6ce-ba484503aa5f admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:29.821419 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 898/1806] 10.222.0.26 () {58 vars in 1152 bytes} [Thu Jul 14 09:18:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 09:18:29.854453 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4ce5ea-ff8d-4948-96c7-10d6f6783f15 tempest-VolumeQuotaClassesTest-1837463828 tempest-VolumeQuotaClassesTest-1837463828-project-admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:29.865685 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 909/1807] 10.222.0.26 () {60 vars in 1152 bytes} [Thu Jul 14 09:18:29 2022] POST /identity/v3/projects => generated 387 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 09:18:29.872181 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4ce5ea-ff8d-4948-96c7-10d6f6783f15 tempest-VolumeQuotaClassesTest-1837463828 tempest-VolumeQuotaClassesTest-1837463828-project-admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:29.891426 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c65103-b571-439b-b6ce-ba484503aa5f admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:29.940769 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:DELETE=4,keystone:SELECT=94,keystone:INSERT=2 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:18:29.944680 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:18:29.967610 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b32cdde-3ecf-43a3-9447-902ed4142119 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:29.968294 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b32cdde-3ecf-43a3-9447-902ed4142119 admin admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:29.968596 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b32cdde-3ecf-43a3-9447-902ed4142119 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:29.968854 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b32cdde-3ecf-43a3-9447-902ed4142119 admin admin] PATH_INFO: `/v3/users` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:29.969554 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b32cdde-3ecf-43a3-9447-902ed4142119 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:29.971697 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b32cdde-3ecf-43a3-9447-902ed4142119 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:29.972137 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c65103-b571-439b-b6ce-ba484503aa5f admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:29.977595 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8255f1bc-bfad-482a-bfa3-28a4a97f5415 tempest-VolumeQuotaClassesTest-1837463828 tempest-VolumeQuotaClassesTest-1837463828-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1907a1379e448108088c3e93e613749', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '714e720078134fbea6740d5c0c7a4bcb', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:29.978668 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8255f1bc-bfad-482a-bfa3-28a4a97f5415 tempest-VolumeQuotaClassesTest-1837463828 tempest-VolumeQuotaClassesTest-1837463828-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:29.979450 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8255f1bc-bfad-482a-bfa3-28a4a97f5415 tempest-VolumeQuotaClassesTest-1837463828 tempest-VolumeQuotaClassesTest-1837463828-project-admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:29.980050 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8255f1bc-bfad-482a-bfa3-28a4a97f5415 tempest-VolumeQuotaClassesTest-1837463828 tempest-VolumeQuotaClassesTest-1837463828-project-admin] PATH_INFO: `/v3/projects/b7f0cb0701f342ff8c0498d87c2dbf50` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:29.982208 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8255f1bc-bfad-482a-bfa3-28a4a97f5415 tempest-VolumeQuotaClassesTest-1837463828 tempest-VolumeQuotaClassesTest-1837463828-project-admin] RBAC: Authorizing `identity:delete_project(project_id=b7f0cb0701f342ff8c0498d87c2dbf50)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:29.984878 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8255f1bc-bfad-482a-bfa3-28a4a97f5415 tempest-VolumeQuotaClassesTest-1837463828 tempest-VolumeQuotaClassesTest-1837463828-project-admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:30.001986 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 899/1808] 10.222.0.26 () {60 vars in 1143 bytes} [Thu Jul 14 09:18:29 2022] POST /identity/v3/users => generated 294 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 09:18:30.006432 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b32cdde-3ecf-43a3-9447-902ed4142119 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:30.021958 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b32cdde-3ecf-43a3-9447-902ed4142119 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:30.028899 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86ddc6f-5616-49f5-bea2-347ddd645513 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:30.030101 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86ddc6f-5616-49f5-bea2-347ddd645513 admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:30.030793 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86ddc6f-5616-49f5-bea2-347ddd645513 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:30.031239 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86ddc6f-5616-49f5-bea2-347ddd645513 admin admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:30.032135 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86ddc6f-5616-49f5-bea2-347ddd645513 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:30.033929 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86ddc6f-5616-49f5-bea2-347ddd645513 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:30.046994 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 900/1809] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jul 14 09:18:30 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 09:18:30.051076 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-8255f1bc-bfad-482a-bfa3-28a4a97f5415 tempest-VolumeQuotaClassesTest-1837463828 tempest-VolumeQuotaClassesTest-1837463828-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b7f0cb0701f342ff8c0498d87c2dbf50', 'request_id': 'req-8255f1bc-bfad-482a-bfa3-28a4a97f5415'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:30.061682 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86ddc6f-5616-49f5-bea2-347ddd645513 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:30.061815 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-8255f1bc-bfad-482a-bfa3-28a4a97f5415 tempest-VolumeQuotaClassesTest-1837463828 tempest-VolumeQuotaClassesTest-1837463828-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b7f0cb0701f342ff8c0498d87c2dbf50', 'request_id': 'req-8255f1bc-bfad-482a-bfa3-28a4a97f5415'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:30.063568 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86ddc6f-5616-49f5-bea2-347ddd645513 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:30.070127 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-8255f1bc-bfad-482a-bfa3-28a4a97f5415 tempest-VolumeQuotaClassesTest-1837463828 tempest-VolumeQuotaClassesTest-1837463828-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b7f0cb0701f342ff8c0498d87c2dbf50', 'request_id': 'req-8255f1bc-bfad-482a-bfa3-28a4a97f5415'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:30.084332 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-8255f1bc-bfad-482a-bfa3-28a4a97f5415 tempest-VolumeQuotaClassesTest-1837463828 tempest-VolumeQuotaClassesTest-1837463828-project-admin] The token cache is being invalidate because project b7f0cb0701f342ff8c0498d87c2dbf50 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80382) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 09:18:30.084627 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-8255f1bc-bfad-482a-bfa3-28a4a97f5415 tempest-VolumeQuotaClassesTest-1837463828 tempest-VolumeQuotaClassesTest-1837463828-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:30.085263 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-8255f1bc-bfad-482a-bfa3-28a4a97f5415 tempest-VolumeQuotaClassesTest-1837463828 tempest-VolumeQuotaClassesTest-1837463828-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:30.087242 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 910/1810] 10.222.0.26 () {58 vars in 1232 bytes} [Thu Jul 14 09:18:29 2022] DELETE /identity/v3/projects/b7f0cb0701f342ff8c0498d87c2dbf50 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:18:30.134251 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48a66bb-365a-4901-b5ae-7cfadda0d69d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:30.135265 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48a66bb-365a-4901-b5ae-7cfadda0d69d admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:30.135554 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48a66bb-365a-4901-b5ae-7cfadda0d69d admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:30.135830 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48a66bb-365a-4901-b5ae-7cfadda0d69d admin admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:30.136564 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48a66bb-365a-4901-b5ae-7cfadda0d69d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:30.138435 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48a66bb-365a-4901-b5ae-7cfadda0d69d admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:30.152656 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 901/1811] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jul 14 09:18:30 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 09:18:30.157256 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8255f1bc-bfad-482a-bfa3-28a4a97f5415 tempest-VolumeQuotaClassesTest-1837463828 tempest-VolumeQuotaClassesTest-1837463828-project-admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:30.159517 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-8255f1bc-bfad-482a-bfa3-28a4a97f5415 tempest-VolumeQuotaClassesTest-1837463828 tempest-VolumeQuotaClassesTest-1837463828-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:18:30.189499 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8255f1bc-bfad-482a-bfa3-28a4a97f5415 tempest-VolumeQuotaClassesTest-1837463828 tempest-VolumeQuotaClassesTest-1837463828-project-admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:30.196896 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8eb2553-8f0e-4946-bf2b-1fa6acadd4c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:30.197922 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8eb2553-8f0e-4946-bf2b-1fa6acadd4c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:30.198267 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8eb2553-8f0e-4946-bf2b-1fa6acadd4c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:30.198610 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8eb2553-8f0e-4946-bf2b-1fa6acadd4c5 admin admin] PATH_INFO: `/v3/projects/46d957e5e80445c7abd21c10958f1b70/users/b51c0680366c4a85a1f4e5d778d8dae1/roles/d1691322ce6a4edab7ce722edf0f2988` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:30.227382 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8eb2553-8f0e-4946-bf2b-1fa6acadd4c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=46d957e5e80445c7abd21c10958f1b70, user_id=b51c0680366c4a85a1f4e5d778d8dae1, role_id=d1691322ce6a4edab7ce722edf0f2988)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:30.231988 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8eb2553-8f0e-4946-bf2b-1fa6acadd4c5 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:30.250789 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 911/1812] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jul 14 09:18:30 2022] PUT /identity/v3/projects/46d957e5e80445c7abd21c10958f1b70/users/b51c0680366c4a85a1f4e5d778d8dae1/roles/d1691322ce6a4edab7ce722edf0f2988 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:18:30.263521 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e91d81f-bbd8-4599-b945-a150af44d55f None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:30.264862 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e91d81f-bbd8-4599-b945-a150af44d55f None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:30.265966 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e91d81f-bbd8-4599-b945-a150af44d55f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:30.315453 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-5e91d81f-bbd8-4599-b945-a150af44d55f None None] MFA Rules not processed for user `b51c0680366c4a85a1f4e5d778d8dae1`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:18:30.320488 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-5e91d81f-bbd8-4599-b945-a150af44d55f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:18:30.384244 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8eb2553-8f0e-4946-bf2b-1fa6acadd4c5 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:30.386095 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-e8eb2553-8f0e-4946-bf2b-1fa6acadd4c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:18:30.394154 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 902/1813] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:18:30 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:18:30.399551 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e91d81f-bbd8-4599-b945-a150af44d55f None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:30.456539 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8eb2553-8f0e-4946-bf2b-1fa6acadd4c5 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:30.462945 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc72229-0c76-4766-860e-83fb85acac8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:30.464031 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc72229-0c76-4766-860e-83fb85acac8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:30.464374 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc72229-0c76-4766-860e-83fb85acac8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:30.464805 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc72229-0c76-4766-860e-83fb85acac8f admin admin] PATH_INFO: `/v3/users/8c7da06d0a334c458fba37e5edbb0c52` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:30.465572 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:18:30.466359 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc72229-0c76-4766-860e-83fb85acac8f admin admin] RBAC: Authorizing `identity:delete_user(user_id=8c7da06d0a334c458fba37e5edbb0c52)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:30.468496 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc72229-0c76-4766-860e-83fb85acac8f admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:30.473361 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e91d81f-bbd8-4599-b945-a150af44d55f None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:30.479606 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dec839-88b8-44d8-b877-9bdcd41199c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:30.480287 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dec839-88b8-44d8-b877-9bdcd41199c5 admin admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:30.480597 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dec839-88b8-44d8-b877-9bdcd41199c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:30.480965 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dec839-88b8-44d8-b877-9bdcd41199c5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:30.481713 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07dec839-88b8-44d8-b877-9bdcd41199c5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:30.483846 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07dec839-88b8-44d8-b877-9bdcd41199c5 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:30.514009 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 903/1814] 10.222.0.26 () {60 vars in 1152 bytes} [Thu Jul 14 09:18:30 2022] POST /identity/v3/projects => generated 387 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 09:18:30.518625 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dec839-88b8-44d8-b877-9bdcd41199c5 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:30.550129 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-acc72229-0c76-4766-860e-83fb85acac8f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8c7da06d0a334c458fba37e5edbb0c52'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:30.551714 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-acc72229-0c76-4766-860e-83fb85acac8f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8c7da06d0a334c458fba37e5edbb0c52'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:30.553249 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:18:30.568134 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-acc72229-0c76-4766-860e-83fb85acac8f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8c7da06d0a334c458fba37e5edbb0c52'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:30.576670 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-acc72229-0c76-4766-860e-83fb85acac8f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8c7da06d0a334c458fba37e5edbb0c52'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:30.577471 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-acc72229-0c76-4766-860e-83fb85acac8f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8c7da06d0a334c458fba37e5edbb0c52'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:30.592500 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 912/1815] 10.222.0.26 () {58 vars in 1223 bytes} [Thu Jul 14 09:18:30 2022] DELETE /identity/v3/users/8c7da06d0a334c458fba37e5edbb0c52 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 09:18:30.597169 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7163ab42-dbcd-42d7-bac1-edf513f1cfea None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:30.597561 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7163ab42-dbcd-42d7-bac1-edf513f1cfea None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:30.597845 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7163ab42-dbcd-42d7-bac1-edf513f1cfea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:30.610460 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dec839-88b8-44d8-b877-9bdcd41199c5 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:30.616374 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-7163ab42-dbcd-42d7-bac1-edf513f1cfea None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:18:30.617781 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-7163ab42-dbcd-42d7-bac1-edf513f1cfea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:18:30.674641 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b50b2e-ed51-454c-88d4-6f817c191501 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:30.675493 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b50b2e-ed51-454c-88d4-6f817c191501 admin admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:30.675968 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b50b2e-ed51-454c-88d4-6f817c191501 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:30.676440 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b50b2e-ed51-454c-88d4-6f817c191501 admin admin] PATH_INFO: `/v3/users` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:30.677386 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b50b2e-ed51-454c-88d4-6f817c191501 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:30.678388 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 913/1816] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:18:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 09:18:30.679699 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b50b2e-ed51-454c-88d4-6f817c191501 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:30.688153 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7163ab42-dbcd-42d7-bac1-edf513f1cfea None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:30.690399 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-7163ab42-dbcd-42d7-bac1-edf513f1cfea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:18:30.704928 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 904/1817] 10.222.0.26 () {60 vars in 1143 bytes} [Thu Jul 14 09:18:30 2022] POST /identity/v3/users => generated 300 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 09:18:30.709134 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b50b2e-ed51-454c-88d4-6f817c191501 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:30.710997 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-03b50b2e-ed51-454c-88d4-6f817c191501 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:18:30.725510 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b50b2e-ed51-454c-88d4-6f817c191501 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:30.734223 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93db054c-8b20-4a22-a1db-f75e305e721d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:30.735018 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93db054c-8b20-4a22-a1db-f75e305e721d admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:30.735452 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93db054c-8b20-4a22-a1db-f75e305e721d admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:30.735848 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93db054c-8b20-4a22-a1db-f75e305e721d admin admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:30.737002 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93db054c-8b20-4a22-a1db-f75e305e721d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:30.738739 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93db054c-8b20-4a22-a1db-f75e305e721d admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:30.753527 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 905/1818] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jul 14 09:18:30 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 09:18:30.754626 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93db054c-8b20-4a22-a1db-f75e305e721d admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:30.761169 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7163ab42-dbcd-42d7-bac1-edf513f1cfea None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:30.765313 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39eefe7f-dcb8-4ca1-9cb1-d45b40e8a408 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '952911c492444ffea4b211a24b564117', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:30.765764 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93db054c-8b20-4a22-a1db-f75e305e721d admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:30.766181 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39eefe7f-dcb8-4ca1-9cb1-d45b40e8a408 service neutron] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:30.766539 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39eefe7f-dcb8-4ca1-9cb1-d45b40e8a408 service neutron] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:30.766822 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39eefe7f-dcb8-4ca1-9cb1-d45b40e8a408 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:30.772170 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc97bac-1cb1-429e-8d65-23b08dd3782f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:30.777751 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39eefe7f-dcb8-4ca1-9cb1-d45b40e8a408 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:30.779379 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39eefe7f-dcb8-4ca1-9cb1-d45b40e8a408 service neutron] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:30.780112 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc97bac-1cb1-429e-8d65-23b08dd3782f admin admin] REQUEST_METHOD: `PUT` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:30.780521 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc97bac-1cb1-429e-8d65-23b08dd3782f admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:30.780895 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc97bac-1cb1-429e-8d65-23b08dd3782f admin admin] PATH_INFO: `/v3/projects/db024195b95444438b884b2be188ad7c/users/919a55b45b254e5d8f8450dc19f199ef/roles/7b549d58d1724395b343609d895c80d0` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:30.791277 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 914/1819] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:18:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 09:18:30.800275 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc97bac-1cb1-429e-8d65-23b08dd3782f admin admin] RBAC: Authorizing `identity:create_grant(project_id=db024195b95444438b884b2be188ad7c, user_id=919a55b45b254e5d8f8450dc19f199ef, role_id=7b549d58d1724395b343609d895c80d0)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:30.803034 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc97bac-1cb1-429e-8d65-23b08dd3782f admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:30.819277 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 906/1820] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jul 14 09:18:30 2022] PUT /identity/v3/projects/db024195b95444438b884b2be188ad7c/users/919a55b45b254e5d8f8450dc19f199ef/roles/7b549d58d1724395b343609d895c80d0 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:18:30.822585 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39eefe7f-dcb8-4ca1-9cb1-d45b40e8a408 service neutron] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:30.835658 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39eefe7f-dcb8-4ca1-9cb1-d45b40e8a408 service neutron] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:30.908710 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad066e4-01c8-4436-bc2a-4fe9157e95fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:30.909423 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad066e4-01c8-4436-bc2a-4fe9157e95fe admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:30.910364 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad066e4-01c8-4436-bc2a-4fe9157e95fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:30.910649 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad066e4-01c8-4436-bc2a-4fe9157e95fe admin admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:30.911424 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad066e4-01c8-4436-bc2a-4fe9157e95fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:30.913078 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad066e4-01c8-4436-bc2a-4fe9157e95fe admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:30.923976 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 915/1821] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jul 14 09:18:30 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 09:18:30.928331 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc97bac-1cb1-429e-8d65-23b08dd3782f admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:30.939160 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc97bac-1cb1-429e-8d65-23b08dd3782f admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:30.943768 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031831f9-4620-4d58-a20e-c8ce1d97f99e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:30.944661 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031831f9-4620-4d58-a20e-c8ce1d97f99e admin admin] REQUEST_METHOD: `PUT` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:30.945026 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031831f9-4620-4d58-a20e-c8ce1d97f99e admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:30.945224 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031831f9-4620-4d58-a20e-c8ce1d97f99e admin admin] PATH_INFO: `/v3/projects/db024195b95444438b884b2be188ad7c/users/919a55b45b254e5d8f8450dc19f199ef/roles/7b549d58d1724395b343609d895c80d0` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:30.947397 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031831f9-4620-4d58-a20e-c8ce1d97f99e admin admin] RBAC: Authorizing `identity:create_grant(project_id=db024195b95444438b884b2be188ad7c, user_id=919a55b45b254e5d8f8450dc19f199ef, role_id=7b549d58d1724395b343609d895c80d0)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:30.950185 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031831f9-4620-4d58-a20e-c8ce1d97f99e admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:30.963244 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 907/1822] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jul 14 09:18:30 2022] PUT /identity/v3/projects/db024195b95444438b884b2be188ad7c/users/919a55b45b254e5d8f8450dc19f199ef/roles/7b549d58d1724395b343609d895c80d0 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 09:18:30.974858 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b08bd3-29a0-40ba-a1ad-c89dd98ea0b3 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:30.975151 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b08bd3-29a0-40ba-a1ad-c89dd98ea0b3 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:30.975488 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b08bd3-29a0-40ba-a1ad-c89dd98ea0b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:30.984442 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:18:31.009651 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-16b08bd3-29a0-40ba-a1ad-c89dd98ea0b3 None None] MFA Rules not processed for user `919a55b45b254e5d8f8450dc19f199ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:18:31.010908 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-16b08bd3-29a0-40ba-a1ad-c89dd98ea0b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:18:31.095907 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 916/1823] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:18:30 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:18:31.107720 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031831f9-4620-4d58-a20e-c8ce1d97f99e admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:31.110052 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-031831f9-4620-4d58-a20e-c8ce1d97f99e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:18:31.172485 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:18:31.217261 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031831f9-4620-4d58-a20e-c8ce1d97f99e admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:31.231350 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7f8fd8-865a-4e9f-a471-628a340fa11e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e76f1952a5b146808844b9f2b7395035', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:31.235355 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7f8fd8-865a-4e9f-a471-628a340fa11e service cinder] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:31.235551 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7f8fd8-865a-4e9f-a471-628a340fa11e service cinder] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:31.235836 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7f8fd8-865a-4e9f-a471-628a340fa11e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:31.256016 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7f8fd8-865a-4e9f-a471-628a340fa11e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:31.258121 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7f8fd8-865a-4e9f-a471-628a340fa11e service cinder] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:31.273141 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 908/1824] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:18:31 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:18:31.409399 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b08bd3-29a0-40ba-a1ad-c89dd98ea0b3 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:31.411620 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-16b08bd3-29a0-40ba-a1ad-c89dd98ea0b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:18:31.427977 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7f8fd8-865a-4e9f-a471-628a340fa11e service cinder] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:31.487500 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b08bd3-29a0-40ba-a1ad-c89dd98ea0b3 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:31.493292 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7100d01-3fcb-44b8-87a2-21accb365279 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:31.494205 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7100d01-3fcb-44b8-87a2-21accb365279 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:31.494490 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7100d01-3fcb-44b8-87a2-21accb365279 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:31.494767 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7100d01-3fcb-44b8-87a2-21accb365279 admin admin] PATH_INFO: `/v3/projects/ddf9a1f821894281bd344b0f28c1dcc4` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:31.496092 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7100d01-3fcb-44b8-87a2-21accb365279 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ddf9a1f821894281bd344b0f28c1dcc4)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:31.498122 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7100d01-3fcb-44b8-87a2-21accb365279 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:31.535160 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7f8fd8-865a-4e9f-a471-628a340fa11e service cinder] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:31.541014 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6eb364-fd29-454a-ac86-24a893a3fa82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:31.542184 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6eb364-fd29-454a-ac86-24a893a3fa82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:31.542672 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6eb364-fd29-454a-ac86-24a893a3fa82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:31.543024 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6eb364-fd29-454a-ac86-24a893a3fa82 admin admin] PATH_INFO: `/v3/users/b51c0680366c4a85a1f4e5d778d8dae1` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:31.544929 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6eb364-fd29-454a-ac86-24a893a3fa82 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b51c0680366c4a85a1f4e5d778d8dae1)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:31.548040 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6eb364-fd29-454a-ac86-24a893a3fa82 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:31.578827 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-c7100d01-3fcb-44b8-87a2-21accb365279 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ddf9a1f821894281bd344b0f28c1dcc4', 'request_id': 'req-c7100d01-3fcb-44b8-87a2-21accb365279'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:31.580838 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-c7100d01-3fcb-44b8-87a2-21accb365279 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ddf9a1f821894281bd344b0f28c1dcc4', 'request_id': 'req-c7100d01-3fcb-44b8-87a2-21accb365279'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:31.589235 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-c7100d01-3fcb-44b8-87a2-21accb365279 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ddf9a1f821894281bd344b0f28c1dcc4', 'request_id': 'req-c7100d01-3fcb-44b8-87a2-21accb365279'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:31.604966 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-cb6eb364-fd29-454a-ac86-24a893a3fa82 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b51c0680366c4a85a1f4e5d778d8dae1'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:31.605118 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-c7100d01-3fcb-44b8-87a2-21accb365279 admin admin] The token cache is being invalidate because project ddf9a1f821894281bd344b0f28c1dcc4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80382) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 09:18:31.605553 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-c7100d01-3fcb-44b8-87a2-21accb365279 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:31.606737 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-c7100d01-3fcb-44b8-87a2-21accb365279 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:31.609616 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 917/1825] 10.222.0.26 () {58 vars in 1232 bytes} [Thu Jul 14 09:18:31 2022] DELETE /identity/v3/projects/ddf9a1f821894281bd344b0f28c1dcc4 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 09:18:31.613858 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7100d01-3fcb-44b8-87a2-21accb365279 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:31.614075 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-cb6eb364-fd29-454a-ac86-24a893a3fa82 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b51c0680366c4a85a1f4e5d778d8dae1'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:31.614990 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-cb6eb364-fd29-454a-ac86-24a893a3fa82 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b51c0680366c4a85a1f4e5d778d8dae1'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:31.615453 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-c7100d01-3fcb-44b8-87a2-21accb365279 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:18:31.615947 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-cb6eb364-fd29-454a-ac86-24a893a3fa82 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b51c0680366c4a85a1f4e5d778d8dae1'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:31.624545 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-cb6eb364-fd29-454a-ac86-24a893a3fa82 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b51c0680366c4a85a1f4e5d778d8dae1'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:31.639126 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:18:31.649307 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 909/1826] 10.222.0.26 () {58 vars in 1223 bytes} [Thu Jul 14 09:18:31 2022] DELETE /identity/v3/users/b51c0680366c4a85a1f4e5d778d8dae1 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 09:18:31.651284 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e406e661-287d-46cd-b5f5-a002114e46ea None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:31.651628 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e406e661-287d-46cd-b5f5-a002114e46ea None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:31.651892 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e406e661-287d-46cd-b5f5-a002114e46ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:31.671341 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-e406e661-287d-46cd-b5f5-a002114e46ea None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:18:31.672688 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-e406e661-287d-46cd-b5f5-a002114e46ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:18:31.690804 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7100d01-3fcb-44b8-87a2-21accb365279 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:31.726031 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:18:31.741179 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 910/1827] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:18:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:18:31.742606 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a30101f-ffcb-43b3-bdf0-fad7632b7b4c None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:31.743004 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a30101f-ffcb-43b3-bdf0-fad7632b7b4c None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:31.743821 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a30101f-ffcb-43b3-bdf0-fad7632b7b4c None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:31.746252 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 911/1828] 10.222.0.26 () {54 vars in 921 bytes} [Thu Jul 14 09:18:31 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:18:31.751090 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a30101f-ffcb-43b3-bdf0-fad7632b7b4c None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:31.752911 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-1a30101f-ffcb-43b3-bdf0-fad7632b7b4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:18:31.760722 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69f1633-703b-4af5-a0b9-e51fb7907487 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:31.761701 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69f1633-703b-4af5-a0b9-e51fb7907487 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:31.762042 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69f1633-703b-4af5-a0b9-e51fb7907487 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:31.762316 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69f1633-703b-4af5-a0b9-e51fb7907487 admin admin] PATH_INFO: `/v3/users/f1907a1379e448108088c3e93e613749` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:31.763576 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69f1633-703b-4af5-a0b9-e51fb7907487 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f1907a1379e448108088c3e93e613749)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:31.765498 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69f1633-703b-4af5-a0b9-e51fb7907487 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:31.818680 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-e69f1633-703b-4af5-a0b9-e51fb7907487 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f1907a1379e448108088c3e93e613749'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:31.819893 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-e69f1633-703b-4af5-a0b9-e51fb7907487 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f1907a1379e448108088c3e93e613749'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:31.828773 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a30101f-ffcb-43b3-bdf0-fad7632b7b4c None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:31.831992 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-e69f1633-703b-4af5-a0b9-e51fb7907487 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f1907a1379e448108088c3e93e613749'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:31.832853 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c36971c-6312-4d24-a2d8-be16b52aa4c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '952911c492444ffea4b211a24b564117', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:31.833611 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c36971c-6312-4d24-a2d8-be16b52aa4c7 service neutron] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:31.834015 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c36971c-6312-4d24-a2d8-be16b52aa4c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:31.834419 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c36971c-6312-4d24-a2d8-be16b52aa4c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:31.836515 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-0c36971c-6312-4d24-a2d8-be16b52aa4c7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:18:31.840504 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-e69f1633-703b-4af5-a0b9-e51fb7907487 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f1907a1379e448108088c3e93e613749'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:31.841148 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-e69f1633-703b-4af5-a0b9-e51fb7907487 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f1907a1379e448108088c3e93e613749'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:31.849563 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c36971c-6312-4d24-a2d8-be16b52aa4c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:31.851220 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c36971c-6312-4d24-a2d8-be16b52aa4c7 service neutron] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:31.854291 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 918/1829] 10.222.0.26 () {58 vars in 1223 bytes} [Thu Jul 14 09:18:31 2022] DELETE /identity/v3/users/f1907a1379e448108088c3e93e613749 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 09:18:31.855760 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2a15c6-23e1-4b52-a921-91093f79d9bc None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:31.856166 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2a15c6-23e1-4b52-a921-91093f79d9bc None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:31.856533 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2a15c6-23e1-4b52-a921-91093f79d9bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:31.865723 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 912/1830] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:18:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:18:31.866419 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c36971c-6312-4d24-a2d8-be16b52aa4c7 service neutron] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:31.867974 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-0c36971c-6312-4d24-a2d8-be16b52aa4c7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:18:31.880838 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c36971c-6312-4d24-a2d8-be16b52aa4c7 service neutron] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:31.881074 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-7d2a15c6-23e1-4b52-a921-91093f79d9bc None None] MFA Rules not processed for user `355684f52e074ad9bfa7fd90bcae6caf`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:18:31.882387 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-7d2a15c6-23e1-4b52-a921-91093f79d9bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:18:31.936911 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:18:31.952215 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6412d7ff-73de-4218-8d42-f68a1dd0e2d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:31.953231 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6412d7ff-73de-4218-8d42-f68a1dd0e2d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:31.953541 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6412d7ff-73de-4218-8d42-f68a1dd0e2d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:31.953810 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6412d7ff-73de-4218-8d42-f68a1dd0e2d5 admin admin] PATH_INFO: `/v3/users/bf084512b5bd4444adfd200e44064741` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:31.955111 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6412d7ff-73de-4218-8d42-f68a1dd0e2d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bf084512b5bd4444adfd200e44064741)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:31.957157 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6412d7ff-73de-4218-8d42-f68a1dd0e2d5 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:31.966592 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 919/1831] 10.222.0.26 () {58 vars in 1018 bytes} [Thu Jul 14 09:18:31 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:18:31.978424 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2a15c6-23e1-4b52-a921-91093f79d9bc None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:31.981055 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-7d2a15c6-23e1-4b52-a921-91093f79d9bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:18:31.997408 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2a15c6-23e1-4b52-a921-91093f79d9bc None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:32.004478 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f20e65-c3bf-492d-bff9-c173f5fcd750 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '355684f52e074ad9bfa7fd90bcae6caf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:32.005591 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f20e65-c3bf-492d-bff9-c173f5fcd750 service placement] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:32.005965 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f20e65-c3bf-492d-bff9-c173f5fcd750 service placement] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:32.006277 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f20e65-c3bf-492d-bff9-c173f5fcd750 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:32.012585 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-6412d7ff-73de-4218-8d42-f68a1dd0e2d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bf084512b5bd4444adfd200e44064741'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:32.017961 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f20e65-c3bf-492d-bff9-c173f5fcd750 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:32.020113 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f20e65-c3bf-492d-bff9-c173f5fcd750 service placement] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:32.022072 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-6412d7ff-73de-4218-8d42-f68a1dd0e2d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bf084512b5bd4444adfd200e44064741'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:32.023086 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-6412d7ff-73de-4218-8d42-f68a1dd0e2d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bf084512b5bd4444adfd200e44064741'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:32.023888 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-6412d7ff-73de-4218-8d42-f68a1dd0e2d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bf084512b5bd4444adfd200e44064741'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:32.031257 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 920/1832] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:18:31 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 09:18:32.034151 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-6412d7ff-73de-4218-8d42-f68a1dd0e2d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bf084512b5bd4444adfd200e44064741'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:32.056076 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 913/1833] 10.222.0.26 () {58 vars in 1223 bytes} [Thu Jul 14 09:18:31 2022] DELETE /identity/v3/users/bf084512b5bd4444adfd200e44064741 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:18:32.349267 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f20e65-c3bf-492d-bff9-c173f5fcd750 service placement] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:32.350755 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-c1f20e65-c3bf-492d-bff9-c173f5fcd750 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:18:32.376958 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f20e65-c3bf-492d-bff9-c173f5fcd750 service placement] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:32.414577 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6412d7ff-73de-4218-8d42-f68a1dd0e2d5 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:32.416116 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-6412d7ff-73de-4218-8d42-f68a1dd0e2d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:18:32.445232 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:18:32.491861 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922459a2-5baf-4895-8a66-3301742d1504 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:32.492692 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922459a2-5baf-4895-8a66-3301742d1504 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:32.492978 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922459a2-5baf-4895-8a66-3301742d1504 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:32.493260 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922459a2-5baf-4895-8a66-3301742d1504 admin admin] PATH_INFO: `/v3/projects/46d957e5e80445c7abd21c10958f1b70` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:32.494493 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922459a2-5baf-4895-8a66-3301742d1504 admin admin] RBAC: Authorizing `identity:delete_project(project_id=46d957e5e80445c7abd21c10958f1b70)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:32.496500 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922459a2-5baf-4895-8a66-3301742d1504 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:32.538333 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6412d7ff-73de-4218-8d42-f68a1dd0e2d5 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:32.544096 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a61960-91a8-43fa-89d2-65a605585f90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:32.545545 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a61960-91a8-43fa-89d2-65a605585f90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:32.545800 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a61960-91a8-43fa-89d2-65a605585f90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:32.546379 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a61960-91a8-43fa-89d2-65a605585f90 admin admin] PATH_INFO: `/v3/projects/714e720078134fbea6740d5c0c7a4bcb` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:32.552506 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a61960-91a8-43fa-89d2-65a605585f90 admin admin] RBAC: Authorizing `identity:delete_project(project_id=714e720078134fbea6740d5c0c7a4bcb)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:32.552506 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a61960-91a8-43fa-89d2-65a605585f90 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:32.574220 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-922459a2-5baf-4895-8a66-3301742d1504 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '46d957e5e80445c7abd21c10958f1b70', 'request_id': 'req-922459a2-5baf-4895-8a66-3301742d1504'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:32.584432 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-922459a2-5baf-4895-8a66-3301742d1504 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '46d957e5e80445c7abd21c10958f1b70', 'request_id': 'req-922459a2-5baf-4895-8a66-3301742d1504'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:32.602948 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-922459a2-5baf-4895-8a66-3301742d1504 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '46d957e5e80445c7abd21c10958f1b70', 'request_id': 'req-922459a2-5baf-4895-8a66-3301742d1504'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:32.632402 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-922459a2-5baf-4895-8a66-3301742d1504 admin admin] The token cache is being invalidate because project 46d957e5e80445c7abd21c10958f1b70 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80382) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 09:18:32.632596 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-922459a2-5baf-4895-8a66-3301742d1504 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:32.633419 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-922459a2-5baf-4895-8a66-3301742d1504 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:32.635762 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 921/1834] 10.222.0.26 () {58 vars in 1232 bytes} [Thu Jul 14 09:18:32 2022] DELETE /identity/v3/projects/46d957e5e80445c7abd21c10958f1b70 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:18:32.640080 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922459a2-5baf-4895-8a66-3301742d1504 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:32.642609 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-922459a2-5baf-4895-8a66-3301742d1504 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:18:32.652651 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-27a61960-91a8-43fa-89d2-65a605585f90 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '714e720078134fbea6740d5c0c7a4bcb', 'request_id': 'req-27a61960-91a8-43fa-89d2-65a605585f90'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:32.663789 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-27a61960-91a8-43fa-89d2-65a605585f90 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '714e720078134fbea6740d5c0c7a4bcb', 'request_id': 'req-27a61960-91a8-43fa-89d2-65a605585f90'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:32.670440 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-27a61960-91a8-43fa-89d2-65a605585f90 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '714e720078134fbea6740d5c0c7a4bcb', 'request_id': 'req-27a61960-91a8-43fa-89d2-65a605585f90'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:32.686552 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-27a61960-91a8-43fa-89d2-65a605585f90 admin admin] The token cache is being invalidate because project 714e720078134fbea6740d5c0c7a4bcb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80381) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 09:18:32.686956 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-27a61960-91a8-43fa-89d2-65a605585f90 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:32.687726 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-27a61960-91a8-43fa-89d2-65a605585f90 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:32.689776 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 914/1835] 10.222.0.26 () {58 vars in 1232 bytes} [Thu Jul 14 09:18:32 2022] DELETE /identity/v3/projects/714e720078134fbea6740d5c0c7a4bcb => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 09:18:32.690669 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a61960-91a8-43fa-89d2-65a605585f90 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:32.692146 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-27a61960-91a8-43fa-89d2-65a605585f90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:18:32.746080 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922459a2-5baf-4895-8a66-3301742d1504 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:32.751236 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:18:32.788288 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a61960-91a8-43fa-89d2-65a605585f90 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:32.817452 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f8659c-6a2d-4ef1-b8ec-918af71241d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:32.820401 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f8659c-6a2d-4ef1-b8ec-918af71241d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:32.821157 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f8659c-6a2d-4ef1-b8ec-918af71241d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:32.821597 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f8659c-6a2d-4ef1-b8ec-918af71241d1 admin admin] PATH_INFO: `/v3/users/919a55b45b254e5d8f8450dc19f199ef` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:32.823993 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f8659c-6a2d-4ef1-b8ec-918af71241d1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=919a55b45b254e5d8f8450dc19f199ef)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:32.826695 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f8659c-6a2d-4ef1-b8ec-918af71241d1 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:32.828708 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c40818-3ad1-4591-b89f-29c146003ea5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:32.829781 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c40818-3ad1-4591-b89f-29c146003ea5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:32.830071 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c40818-3ad1-4591-b89f-29c146003ea5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:32.830353 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c40818-3ad1-4591-b89f-29c146003ea5 admin admin] PATH_INFO: `/v3/projects/db823124e7704309be3370a64fcde324` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:32.831604 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c40818-3ad1-4591-b89f-29c146003ea5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=db823124e7704309be3370a64fcde324)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:32.833538 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c40818-3ad1-4591-b89f-29c146003ea5 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:32.895876 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-78f8659c-6a2d-4ef1-b8ec-918af71241d1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '919a55b45b254e5d8f8450dc19f199ef'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:32.897882 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-e3c40818-3ad1-4591-b89f-29c146003ea5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'db823124e7704309be3370a64fcde324', 'request_id': 'req-e3c40818-3ad1-4591-b89f-29c146003ea5'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:32.898650 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:18:32.905779 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-e3c40818-3ad1-4591-b89f-29c146003ea5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'db823124e7704309be3370a64fcde324', 'request_id': 'req-e3c40818-3ad1-4591-b89f-29c146003ea5'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:32.911349 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-78f8659c-6a2d-4ef1-b8ec-918af71241d1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '919a55b45b254e5d8f8450dc19f199ef'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:32.912680 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-78f8659c-6a2d-4ef1-b8ec-918af71241d1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '919a55b45b254e5d8f8450dc19f199ef'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:32.914168 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-78f8659c-6a2d-4ef1-b8ec-918af71241d1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '919a55b45b254e5d8f8450dc19f199ef'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:32.915049 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-e3c40818-3ad1-4591-b89f-29c146003ea5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'db823124e7704309be3370a64fcde324', 'request_id': 'req-e3c40818-3ad1-4591-b89f-29c146003ea5'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:32.926777 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-78f8659c-6a2d-4ef1-b8ec-918af71241d1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '919a55b45b254e5d8f8450dc19f199ef'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:32.941223 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-e3c40818-3ad1-4591-b89f-29c146003ea5 admin admin] The token cache is being invalidate because project db823124e7704309be3370a64fcde324 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80382) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 09:18:32.941223 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-e3c40818-3ad1-4591-b89f-29c146003ea5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:32.941223 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-e3c40818-3ad1-4591-b89f-29c146003ea5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:32.943677 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 922/1836] 10.222.0.26 () {58 vars in 1232 bytes} [Thu Jul 14 09:18:32 2022] DELETE /identity/v3/projects/db823124e7704309be3370a64fcde324 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:18:32.945197 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43e0082-8c9a-4331-9af3-9950b5718a68 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:32.945492 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43e0082-8c9a-4331-9af3-9950b5718a68 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:32.945777 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43e0082-8c9a-4331-9af3-9950b5718a68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:32.966203 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 915/1837] 10.222.0.26 () {58 vars in 1223 bytes} [Thu Jul 14 09:18:32 2022] DELETE /identity/v3/users/919a55b45b254e5d8f8450dc19f199ef => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:18:32.967107 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f8659c-6a2d-4ef1-b8ec-918af71241d1 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:32.973458 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-78f8659c-6a2d-4ef1-b8ec-918af71241d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:18:32.978288 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-f43e0082-8c9a-4331-9af3-9950b5718a68 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:18:32.979646 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-f43e0082-8c9a-4331-9af3-9950b5718a68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:18:33.079352 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 923/1838] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:18:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:18:33.087811 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbce2f4-b167-4957-975f-3d46242943d7 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:33.088234 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbce2f4-b167-4957-975f-3d46242943d7 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:33.088521 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbce2f4-b167-4957-975f-3d46242943d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:33.102562 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f8659c-6a2d-4ef1-b8ec-918af71241d1 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:33.108987 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a07f9f-dab5-485d-8347-7653a11318bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:33.110326 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a07f9f-dab5-485d-8347-7653a11318bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:33.110767 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a07f9f-dab5-485d-8347-7653a11318bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:33.111140 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a07f9f-dab5-485d-8347-7653a11318bd admin admin] PATH_INFO: `/v3/users/27b9db8dc2e34872b2a39bb4d527c5e8` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:33.112858 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-4dbce2f4-b167-4957-975f-3d46242943d7 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:18:33.113025 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a07f9f-dab5-485d-8347-7653a11318bd admin admin] RBAC: Authorizing `identity:delete_user(user_id=27b9db8dc2e34872b2a39bb4d527c5e8)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:33.114348 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-4dbce2f4-b167-4957-975f-3d46242943d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:18:33.115544 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a07f9f-dab5-485d-8347-7653a11318bd admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:33.126280 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 924/1839] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:18:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:18:33.131286 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbce2f4-b167-4957-975f-3d46242943d7 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:33.140802 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbce2f4-b167-4957-975f-3d46242943d7 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:33.146910 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e504d0-5e1f-4022-b271-c1c3c3e056e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:33.147693 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e504d0-5e1f-4022-b271-c1c3c3e056e7 admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:33.147979 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e504d0-5e1f-4022-b271-c1c3c3e056e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:33.148289 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e504d0-5e1f-4022-b271-c1c3c3e056e7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:33.149125 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e504d0-5e1f-4022-b271-c1c3c3e056e7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:18:33.149629 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e504d0-5e1f-4022-b271-c1c3c3e056e7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:33.151598 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e504d0-5e1f-4022-b271-c1c3c3e056e7 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:33.168132 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 925/1840] 10.222.0.26 () {58 vars in 1152 bytes} [Thu Jul 14 09:18:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:18:33.172812 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e504d0-5e1f-4022-b271-c1c3c3e056e7 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:33.181216 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-06a07f9f-dab5-485d-8347-7653a11318bd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '27b9db8dc2e34872b2a39bb4d527c5e8'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:33.182656 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e504d0-5e1f-4022-b271-c1c3c3e056e7 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:33.188576 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b36817-059f-481c-9440-dce568d04f9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:33.189286 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b36817-059f-481c-9440-dce568d04f9c admin admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:33.189609 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b36817-059f-481c-9440-dce568d04f9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:33.189963 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b36817-059f-481c-9440-dce568d04f9c admin admin] PATH_INFO: `/v3/projects` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:33.190590 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b36817-059f-481c-9440-dce568d04f9c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:33.191627 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-06a07f9f-dab5-485d-8347-7653a11318bd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '27b9db8dc2e34872b2a39bb4d527c5e8'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:33.192619 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b36817-059f-481c-9440-dce568d04f9c admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:33.193255 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-06a07f9f-dab5-485d-8347-7653a11318bd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '27b9db8dc2e34872b2a39bb4d527c5e8'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:33.194288 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-06a07f9f-dab5-485d-8347-7653a11318bd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '27b9db8dc2e34872b2a39bb4d527c5e8'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:33.203706 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-06a07f9f-dab5-485d-8347-7653a11318bd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '27b9db8dc2e34872b2a39bb4d527c5e8'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:33.208795 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:18:33.230107 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 926/1841] 10.222.0.26 () {60 vars in 1152 bytes} [Thu Jul 14 09:18:33 2022] POST /identity/v3/projects => generated 393 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 09:18:33.230298 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 916/1842] 10.222.0.26 () {58 vars in 1223 bytes} [Thu Jul 14 09:18:32 2022] DELETE /identity/v3/users/27b9db8dc2e34872b2a39bb4d527c5e8 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:18:33.231175 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b36817-059f-481c-9440-dce568d04f9c admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:33.232954 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-46b36817-059f-481c-9440-dce568d04f9c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:18:33.300776 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b36817-059f-481c-9440-dce568d04f9c admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:33.350010 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a07f9f-dab5-485d-8347-7653a11318bd admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:33.351717 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-06a07f9f-dab5-485d-8347-7653a11318bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:18:33.418093 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4b8a71-bd2e-4fc7-922f-32e5909112e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:33.418914 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4b8a71-bd2e-4fc7-922f-32e5909112e1 admin admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:33.419294 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4b8a71-bd2e-4fc7-922f-32e5909112e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:33.419548 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4b8a71-bd2e-4fc7-922f-32e5909112e1 admin admin] PATH_INFO: `/v3/users` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:33.420265 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4b8a71-bd2e-4fc7-922f-32e5909112e1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:33.422264 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4b8a71-bd2e-4fc7-922f-32e5909112e1 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:33.457124 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 927/1843] 10.222.0.26 () {60 vars in 1143 bytes} [Thu Jul 14 09:18:33 2022] POST /identity/v3/users => generated 297 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 09:18:33.461756 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4b8a71-bd2e-4fc7-922f-32e5909112e1 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:33.476510 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4b8a71-bd2e-4fc7-922f-32e5909112e1 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:33.481936 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb85e95d-f5f8-4404-b2d5-338aecb4958c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:33.482700 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb85e95d-f5f8-4404-b2d5-338aecb4958c admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:33.483044 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb85e95d-f5f8-4404-b2d5-338aecb4958c admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:33.483981 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb85e95d-f5f8-4404-b2d5-338aecb4958c admin admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:33.484995 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb85e95d-f5f8-4404-b2d5-338aecb4958c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:33.486807 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb85e95d-f5f8-4404-b2d5-338aecb4958c admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:33.488972 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:INSERT=2,keystone:DELETE=13,keystone:SELECT=85 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:18:33.501023 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 928/1844] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jul 14 09:18:33 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 09:18:33.505368 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb85e95d-f5f8-4404-b2d5-338aecb4958c admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:33.527021 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb85e95d-f5f8-4404-b2d5-338aecb4958c admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:33.538416 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3c99e8-383b-465d-9c98-712859551b25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:33.540258 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3c99e8-383b-465d-9c98-712859551b25 admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:33.540659 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3c99e8-383b-465d-9c98-712859551b25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:33.541447 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3c99e8-383b-465d-9c98-712859551b25 admin admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:33.542052 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3c99e8-383b-465d-9c98-712859551b25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:33.552516 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3c99e8-383b-465d-9c98-712859551b25 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:33.561753 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a07f9f-dab5-485d-8347-7653a11318bd admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:33.573334 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34ec7da-55a3-41f5-ba9e-412ee9702dfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:33.574568 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34ec7da-55a3-41f5-ba9e-412ee9702dfe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:33.574925 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34ec7da-55a3-41f5-ba9e-412ee9702dfe admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:33.575302 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34ec7da-55a3-41f5-ba9e-412ee9702dfe admin admin] PATH_INFO: `/v3/projects/db024195b95444438b884b2be188ad7c` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:33.576727 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34ec7da-55a3-41f5-ba9e-412ee9702dfe admin admin] RBAC: Authorizing `identity:delete_project(project_id=db024195b95444438b884b2be188ad7c)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:33.578740 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34ec7da-55a3-41f5-ba9e-412ee9702dfe admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:33.595760 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 929/1845] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jul 14 09:18:33 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 09:18:33.596916 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3c99e8-383b-465d-9c98-712859551b25 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:33.599523 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-6e3c99e8-383b-465d-9c98-712859551b25 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:18:33.618700 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3c99e8-383b-465d-9c98-712859551b25 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:33.626412 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464878c9-1b0a-4b56-8fea-cb972841a78a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:33.627714 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464878c9-1b0a-4b56-8fea-cb972841a78a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:33.628307 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464878c9-1b0a-4b56-8fea-cb972841a78a admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:33.633268 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464878c9-1b0a-4b56-8fea-cb972841a78a admin admin] PATH_INFO: `/v3/projects/ac934a31c1364a4f8231d8eaa11044eb` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:33.636553 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464878c9-1b0a-4b56-8fea-cb972841a78a admin admin] RBAC: Authorizing `identity:delete_project(project_id=ac934a31c1364a4f8231d8eaa11044eb)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:33.638828 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464878c9-1b0a-4b56-8fea-cb972841a78a admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:33.680679 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-e34ec7da-55a3-41f5-ba9e-412ee9702dfe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'db024195b95444438b884b2be188ad7c', 'request_id': 'req-e34ec7da-55a3-41f5-ba9e-412ee9702dfe'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:33.688276 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-e34ec7da-55a3-41f5-ba9e-412ee9702dfe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'db024195b95444438b884b2be188ad7c', 'request_id': 'req-e34ec7da-55a3-41f5-ba9e-412ee9702dfe'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:33.695258 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-e34ec7da-55a3-41f5-ba9e-412ee9702dfe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'db024195b95444438b884b2be188ad7c', 'request_id': 'req-e34ec7da-55a3-41f5-ba9e-412ee9702dfe'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:33.708757 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-464878c9-1b0a-4b56-8fea-cb972841a78a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ac934a31c1364a4f8231d8eaa11044eb', 'request_id': 'req-464878c9-1b0a-4b56-8fea-cb972841a78a'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:33.714352 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-e34ec7da-55a3-41f5-ba9e-412ee9702dfe admin admin] The token cache is being invalidate because project db024195b95444438b884b2be188ad7c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80381) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 09:18:33.714649 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-e34ec7da-55a3-41f5-ba9e-412ee9702dfe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:33.715525 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-e34ec7da-55a3-41f5-ba9e-412ee9702dfe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:33.716342 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-464878c9-1b0a-4b56-8fea-cb972841a78a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ac934a31c1364a4f8231d8eaa11044eb', 'request_id': 'req-464878c9-1b0a-4b56-8fea-cb972841a78a'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:33.720697 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 917/1846] 10.222.0.26 () {58 vars in 1232 bytes} [Thu Jul 14 09:18:33 2022] DELETE /identity/v3/projects/db024195b95444438b884b2be188ad7c => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 09:18:33.720697 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34ec7da-55a3-41f5-ba9e-412ee9702dfe admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:33.722313 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-e34ec7da-55a3-41f5-ba9e-412ee9702dfe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:18:33.731854 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-464878c9-1b0a-4b56-8fea-cb972841a78a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ac934a31c1364a4f8231d8eaa11044eb', 'request_id': 'req-464878c9-1b0a-4b56-8fea-cb972841a78a'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:33.737351 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=90,keystone:INSERT=4,keystone:DELETE=6 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:18:33.757690 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-464878c9-1b0a-4b56-8fea-cb972841a78a admin admin] The token cache is being invalidate because project ac934a31c1364a4f8231d8eaa11044eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80382) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 09:18:33.757854 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-464878c9-1b0a-4b56-8fea-cb972841a78a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:33.759490 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-464878c9-1b0a-4b56-8fea-cb972841a78a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:33.762422 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 930/1847] 10.222.0.26 () {58 vars in 1232 bytes} [Thu Jul 14 09:18:33 2022] DELETE /identity/v3/projects/ac934a31c1364a4f8231d8eaa11044eb => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:18:33.769169 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc8412e-6365-4bfb-bf41-d9deff9ffa00 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:33.769887 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc8412e-6365-4bfb-bf41-d9deff9ffa00 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:33.770953 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc8412e-6365-4bfb-bf41-d9deff9ffa00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:33.797200 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-3cc8412e-6365-4bfb-bf41-d9deff9ffa00 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:18:33.799061 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-3cc8412e-6365-4bfb-bf41-d9deff9ffa00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:18:33.838259 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34ec7da-55a3-41f5-ba9e-412ee9702dfe admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:33.909381 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 931/1848] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:18:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:18:33.917145 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecb4d0d-17ab-4a07-9572-d9996928d32c None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:33.917449 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecb4d0d-17ab-4a07-9572-d9996928d32c None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:33.917726 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecb4d0d-17ab-4a07-9572-d9996928d32c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:33.935760 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f059d2b-edc7-4d1f-9233-a5a895d4ebb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:33.936800 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f059d2b-edc7-4d1f-9233-a5a895d4ebb5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:33.937254 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f059d2b-edc7-4d1f-9233-a5a895d4ebb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:33.937630 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f059d2b-edc7-4d1f-9233-a5a895d4ebb5 admin admin] PATH_INFO: `/v3/projects/c3b4991d3836498383beee96b2c6507c/users/df88a63188cf4ab0b7e24050a83444c5/roles/d1691322ce6a4edab7ce722edf0f2988` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:33.947680 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-2ecb4d0d-17ab-4a07-9572-d9996928d32c None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:18:33.948905 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-2ecb4d0d-17ab-4a07-9572-d9996928d32c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:18:33.959811 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 932/1849] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:18:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:18:33.961825 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7000d8ec-58f4-45a8-a169-413746855a4c None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:33.962186 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7000d8ec-58f4-45a8-a169-413746855a4c None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:33.962845 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7000d8ec-58f4-45a8-a169-413746855a4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:33.974376 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f059d2b-edc7-4d1f-9233-a5a895d4ebb5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3b4991d3836498383beee96b2c6507c, user_id=df88a63188cf4ab0b7e24050a83444c5, role_id=d1691322ce6a4edab7ce722edf0f2988)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:33.977216 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f059d2b-edc7-4d1f-9233-a5a895d4ebb5 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:33.987297 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-7000d8ec-58f4-45a8-a169-413746855a4c None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:18:33.989043 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-7000d8ec-58f4-45a8-a169-413746855a4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:18:34.000710 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 918/1850] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jul 14 09:18:33 2022] PUT /identity/v3/projects/c3b4991d3836498383beee96b2c6507c/users/df88a63188cf4ab0b7e24050a83444c5/roles/d1691322ce6a4edab7ce722edf0f2988 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:18:34.002464 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc92edb-a1cf-44ca-aa83-fa116716f47f None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:34.002775 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc92edb-a1cf-44ca-aa83-fa116716f47f None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:34.003149 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc92edb-a1cf-44ca-aa83-fa116716f47f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:34.013959 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:18:34.030088 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-1bc92edb-a1cf-44ca-aa83-fa116716f47f None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:18:34.031728 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-1bc92edb-a1cf-44ca-aa83-fa116716f47f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:18:34.085017 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 933/1851] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:18:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:18:34.086344 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f0b902-efe3-4393-874e-7ee6fcfea371 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:34.086632 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f0b902-efe3-4393-874e-7ee6fcfea371 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:34.086921 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f0b902-efe3-4393-874e-7ee6fcfea371 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:34.117491 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-86f0b902-efe3-4393-874e-7ee6fcfea371 None None] MFA Rules not processed for user `df88a63188cf4ab0b7e24050a83444c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:18:34.118767 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-86f0b902-efe3-4393-874e-7ee6fcfea371 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:18:34.122894 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 919/1852] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:18:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:18:34.123855 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc92edb-a1cf-44ca-aa83-fa116716f47f None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:34.133500 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc92edb-a1cf-44ca-aa83-fa116716f47f None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:34.138953 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309a83bc-d06f-4b1c-88b8-311175447d2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:34.139651 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309a83bc-d06f-4b1c-88b8-311175447d2a admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:34.139937 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309a83bc-d06f-4b1c-88b8-311175447d2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:34.140217 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309a83bc-d06f-4b1c-88b8-311175447d2a admin admin] PATH_INFO: `/v3/domains` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:34.140949 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309a83bc-d06f-4b1c-88b8-311175447d2a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:18:34.141258 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309a83bc-d06f-4b1c-88b8-311175447d2a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:34.142860 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309a83bc-d06f-4b1c-88b8-311175447d2a admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:34.156201 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 920/1853] 10.222.0.26 () {58 vars in 1152 bytes} [Thu Jul 14 09:18:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:18:34.156873 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309a83bc-d06f-4b1c-88b8-311175447d2a admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:34.166828 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309a83bc-d06f-4b1c-88b8-311175447d2a admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:34.171663 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc14fb3-a7c8-46a2-a21a-4511c6b9a0d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:34.172325 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc14fb3-a7c8-46a2-a21a-4511c6b9a0d8 admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:34.172610 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc14fb3-a7c8-46a2-a21a-4511c6b9a0d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:34.172889 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc14fb3-a7c8-46a2-a21a-4511c6b9a0d8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:34.173618 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc14fb3-a7c8-46a2-a21a-4511c6b9a0d8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:18:34.173934 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc14fb3-a7c8-46a2-a21a-4511c6b9a0d8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:34.175744 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc14fb3-a7c8-46a2-a21a-4511c6b9a0d8 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:34.190251 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 921/1854] 10.222.0.26 () {58 vars in 1152 bytes} [Thu Jul 14 09:18:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:18:34.191122 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc14fb3-a7c8-46a2-a21a-4511c6b9a0d8 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:34.201972 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc14fb3-a7c8-46a2-a21a-4511c6b9a0d8 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:34.207533 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 934/1855] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:18:34 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:18:34.207533 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f0b902-efe3-4393-874e-7ee6fcfea371 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:34.211240 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00be7289-a156-430b-b061-2bb2f1126d45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:34.212722 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00be7289-a156-430b-b061-2bb2f1126d45 admin admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:34.213014 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00be7289-a156-430b-b061-2bb2f1126d45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:34.213476 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00be7289-a156-430b-b061-2bb2f1126d45 admin admin] PATH_INFO: `/v3/projects` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:34.214390 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00be7289-a156-430b-b061-2bb2f1126d45 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:34.216562 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00be7289-a156-430b-b061-2bb2f1126d45 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:34.218769 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f0b902-efe3-4393-874e-7ee6fcfea371 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:34.224599 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729f6b5f-daa7-49c2-8ce9-27a55ed81774 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:34.225526 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729f6b5f-daa7-49c2-8ce9-27a55ed81774 admin admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:34.225908 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729f6b5f-daa7-49c2-8ce9-27a55ed81774 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:34.226290 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729f6b5f-daa7-49c2-8ce9-27a55ed81774 admin admin] PATH_INFO: `/v3/projects` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:34.227472 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729f6b5f-daa7-49c2-8ce9-27a55ed81774 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:34.230289 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729f6b5f-daa7-49c2-8ce9-27a55ed81774 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:34.247678 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:DELETE=1,keystone:SELECT=98,keystone:INSERT=1 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:18:34.249258 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 922/1856] 10.222.0.26 () {60 vars in 1152 bytes} [Thu Jul 14 09:18:34 2022] POST /identity/v3/projects => generated 367 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 09:18:34.250072 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00be7289-a156-430b-b061-2bb2f1126d45 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:34.253332 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-00be7289-a156-430b-b061-2bb2f1126d45 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:18:34.266557 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 935/1857] 10.222.0.26 () {60 vars in 1152 bytes} [Thu Jul 14 09:18:34 2022] POST /identity/v3/projects => generated 383 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 09:18:34.267517 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729f6b5f-daa7-49c2-8ce9-27a55ed81774 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:34.333221 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00be7289-a156-430b-b061-2bb2f1126d45 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:34.351924 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729f6b5f-daa7-49c2-8ce9-27a55ed81774 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:34.357746 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0092e13e-7b38-4186-8477-ff5f4f52085d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:34.358632 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0092e13e-7b38-4186-8477-ff5f4f52085d admin admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:34.358938 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0092e13e-7b38-4186-8477-ff5f4f52085d admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:34.359237 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0092e13e-7b38-4186-8477-ff5f4f52085d admin admin] PATH_INFO: `/v3/users` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:34.360202 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0092e13e-7b38-4186-8477-ff5f4f52085d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:34.362721 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0092e13e-7b38-4186-8477-ff5f4f52085d admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:34.389590 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 936/1858] 10.222.0.26 () {60 vars in 1143 bytes} [Thu Jul 14 09:18:34 2022] POST /identity/v3/users => generated 284 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 09:18:34.390557 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0092e13e-7b38-4186-8477-ff5f4f52085d admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:34.403894 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0092e13e-7b38-4186-8477-ff5f4f52085d admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:34.405088 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c69fea-77e7-4ee5-8283-ab5aa8e84ab0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:34.405797 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c69fea-77e7-4ee5-8283-ab5aa8e84ab0 admin admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:34.406111 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c69fea-77e7-4ee5-8283-ab5aa8e84ab0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:34.406577 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c69fea-77e7-4ee5-8283-ab5aa8e84ab0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:34.407436 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c69fea-77e7-4ee5-8283-ab5aa8e84ab0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:34.408841 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4ad852-093c-4975-8b69-8d33251bbc6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:34.409499 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c69fea-77e7-4ee5-8283-ab5aa8e84ab0 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:34.409626 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4ad852-093c-4975-8b69-8d33251bbc6e admin admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:34.409833 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4ad852-093c-4975-8b69-8d33251bbc6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:34.410108 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4ad852-093c-4975-8b69-8d33251bbc6e admin admin] PATH_INFO: `/v3/users` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:34.410825 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4ad852-093c-4975-8b69-8d33251bbc6e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:34.412752 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4ad852-093c-4975-8b69-8d33251bbc6e admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:34.419314 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:18:34.437096 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 937/1859] 10.222.0.26 () {60 vars in 1143 bytes} [Thu Jul 14 09:18:34 2022] POST /identity/v3/users => generated 292 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 09:18:34.437096 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4ad852-093c-4975-8b69-8d33251bbc6e admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:34.437971 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 923/1860] 10.222.0.26 () {60 vars in 1152 bytes} [Thu Jul 14 09:18:34 2022] POST /identity/v3/projects => generated 391 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 09:18:34.439038 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c69fea-77e7-4ee5-8283-ab5aa8e84ab0 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:34.444999 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4ad852-093c-4975-8b69-8d33251bbc6e admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:34.503740 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1655df6-2663-4c33-b2e7-20cf25b07862 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:34.504466 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1655df6-2663-4c33-b2e7-20cf25b07862 admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:34.504752 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1655df6-2663-4c33-b2e7-20cf25b07862 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:34.505031 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1655df6-2663-4c33-b2e7-20cf25b07862 admin admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:34.505776 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1655df6-2663-4c33-b2e7-20cf25b07862 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:34.506594 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c69fea-77e7-4ee5-8283-ab5aa8e84ab0 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:34.507555 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1655df6-2663-4c33-b2e7-20cf25b07862 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:34.511671 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71325e4e-e743-425c-b3d8-85fdbcbd8bc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:34.512449 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71325e4e-e743-425c-b3d8-85fdbcbd8bc8 admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:34.512737 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71325e4e-e743-425c-b3d8-85fdbcbd8bc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:34.513015 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71325e4e-e743-425c-b3d8-85fdbcbd8bc8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:34.513748 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71325e4e-e743-425c-b3d8-85fdbcbd8bc8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:34.515425 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71325e4e-e743-425c-b3d8-85fdbcbd8bc8 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:34.518610 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 938/1861] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jul 14 09:18:34 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 09:18:34.519386 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1655df6-2663-4c33-b2e7-20cf25b07862 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:34.528129 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 924/1862] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jul 14 09:18:34 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 09:18:34.529563 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71325e4e-e743-425c-b3d8-85fdbcbd8bc8 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:34.530965 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1655df6-2663-4c33-b2e7-20cf25b07862 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:34.539749 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8af37f-f157-456e-be85-c6c719681a39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:34.540394 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71325e4e-e743-425c-b3d8-85fdbcbd8bc8 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:34.540528 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8af37f-f157-456e-be85-c6c719681a39 admin admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:34.540695 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8af37f-f157-456e-be85-c6c719681a39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:34.540937 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8af37f-f157-456e-be85-c6c719681a39 admin admin] PATH_INFO: `/v3/users` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:34.541983 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8af37f-f157-456e-be85-c6c719681a39 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:34.543771 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8af37f-f157-456e-be85-c6c719681a39 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:34.545382 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71d3a2b-e1a3-4f77-912a-0db61b55bdb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:34.546476 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71d3a2b-e1a3-4f77-912a-0db61b55bdb1 admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:34.547341 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71d3a2b-e1a3-4f77-912a-0db61b55bdb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:34.547341 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71d3a2b-e1a3-4f77-912a-0db61b55bdb1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:34.547851 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71d3a2b-e1a3-4f77-912a-0db61b55bdb1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:34.549451 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71d3a2b-e1a3-4f77-912a-0db61b55bdb1 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:34.562582 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 925/1863] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jul 14 09:18:34 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 09:18:34.564333 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71d3a2b-e1a3-4f77-912a-0db61b55bdb1 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:34.567957 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 939/1864] 10.222.0.26 () {60 vars in 1143 bytes} [Thu Jul 14 09:18:34 2022] POST /identity/v3/users => generated 302 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 09:18:34.568774 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8af37f-f157-456e-be85-c6c719681a39 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:34.576007 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71d3a2b-e1a3-4f77-912a-0db61b55bdb1 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:34.580118 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8af37f-f157-456e-be85-c6c719681a39 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:34.581479 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1224fb18-7e9a-46d0-8140-1466f1f45341 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:34.582143 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1224fb18-7e9a-46d0-8140-1466f1f45341 admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:34.582430 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1224fb18-7e9a-46d0-8140-1466f1f45341 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:34.582708 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1224fb18-7e9a-46d0-8140-1466f1f45341 admin admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:34.583493 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1224fb18-7e9a-46d0-8140-1466f1f45341 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:34.584942 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6409e50-a798-480d-9f3b-b1411b67f8a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:34.585103 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1224fb18-7e9a-46d0-8140-1466f1f45341 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:34.585823 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6409e50-a798-480d-9f3b-b1411b67f8a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:34.586169 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6409e50-a798-480d-9f3b-b1411b67f8a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:34.586576 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6409e50-a798-480d-9f3b-b1411b67f8a5 admin admin] PATH_INFO: `/v3/projects/b84d87c0145a4d3dbe5435d9ce225e3e/users/1435cdf59c774ea3b091da8723e6cbff/roles/d1691322ce6a4edab7ce722edf0f2988` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:34.595891 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 926/1865] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jul 14 09:18:34 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 09:18:34.596660 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1224fb18-7e9a-46d0-8140-1466f1f45341 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:34.604020 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6409e50-a798-480d-9f3b-b1411b67f8a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b84d87c0145a4d3dbe5435d9ce225e3e, user_id=1435cdf59c774ea3b091da8723e6cbff, role_id=d1691322ce6a4edab7ce722edf0f2988)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:34.606348 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1224fb18-7e9a-46d0-8140-1466f1f45341 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:34.606877 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6409e50-a798-480d-9f3b-b1411b67f8a5 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:34.611471 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e939425e-45d9-4166-8fbf-c6c6e580d88b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:34.612225 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e939425e-45d9-4166-8fbf-c6c6e580d88b admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:34.612660 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e939425e-45d9-4166-8fbf-c6c6e580d88b admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:34.612796 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e939425e-45d9-4166-8fbf-c6c6e580d88b admin admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:34.613532 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e939425e-45d9-4166-8fbf-c6c6e580d88b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:34.615229 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e939425e-45d9-4166-8fbf-c6c6e580d88b admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:34.620805 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 940/1866] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jul 14 09:18:34 2022] PUT /identity/v3/projects/b84d87c0145a4d3dbe5435d9ce225e3e/users/1435cdf59c774ea3b091da8723e6cbff/roles/d1691322ce6a4edab7ce722edf0f2988 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:18:34.629752 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6409e50-a798-480d-9f3b-b1411b67f8a5 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:34.629884 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 927/1867] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jul 14 09:18:34 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 09:18:34.631123 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f43b4e3-8425-421d-a87c-fb64597f8842 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:34.631494 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f43b4e3-8425-421d-a87c-fb64597f8842 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:34.631896 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f43b4e3-8425-421d-a87c-fb64597f8842 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:34.661151 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:18:34.666684 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-7f43b4e3-8425-421d-a87c-fb64597f8842 None None] MFA Rules not processed for user `1435cdf59c774ea3b091da8723e6cbff`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:18:34.669873 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-7f43b4e3-8425-421d-a87c-fb64597f8842 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:18:34.709922 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6409e50-a798-480d-9f3b-b1411b67f8a5 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:34.715561 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0d8bcb-a2d7-46f5-80db-52538bc67be6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:34.716660 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0d8bcb-a2d7-46f5-80db-52538bc67be6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:34.717044 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0d8bcb-a2d7-46f5-80db-52538bc67be6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:34.717520 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0d8bcb-a2d7-46f5-80db-52538bc67be6 admin admin] PATH_INFO: `/v3/projects/9e608162ad754d92b7dbcb9f7eb99368/users/b4eb9317307e4ebebe4d430ef757bb46/roles/d1691322ce6a4edab7ce722edf0f2988` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:34.734079 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0d8bcb-a2d7-46f5-80db-52538bc67be6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e608162ad754d92b7dbcb9f7eb99368, user_id=b4eb9317307e4ebebe4d430ef757bb46, role_id=d1691322ce6a4edab7ce722edf0f2988)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:34.737058 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0d8bcb-a2d7-46f5-80db-52538bc67be6 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:34.744627 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 928/1868] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:18:34 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 09:18:34.748317 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f43b4e3-8425-421d-a87c-fb64597f8842 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:34.758109 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 941/1869] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jul 14 09:18:34 2022] PUT /identity/v3/projects/9e608162ad754d92b7dbcb9f7eb99368/users/b4eb9317307e4ebebe4d430ef757bb46/roles/d1691322ce6a4edab7ce722edf0f2988 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:18:34.758723 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0d8bcb-a2d7-46f5-80db-52538bc67be6 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:34.759681 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f43b4e3-8425-421d-a87c-fb64597f8842 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:34.820407 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:18:34.845152 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b68e16-b770-4084-96cc-55ff3867325f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:34.846611 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b68e16-b770-4084-96cc-55ff3867325f admin admin] REQUEST_METHOD: `PUT` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:34.846901 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b68e16-b770-4084-96cc-55ff3867325f admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:34.847798 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b68e16-b770-4084-96cc-55ff3867325f admin admin] PATH_INFO: `/v3/projects/2dfa1d5e59664af5a450bb1003ffbb4d/users/d15970cea431479f8b0aec00863efa48/roles/7b549d58d1724395b343609d895c80d0` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:34.852388 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0d8bcb-a2d7-46f5-80db-52538bc67be6 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:34.857700 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b501b3-dee9-405f-9a50-69602c801cea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:34.858676 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b501b3-dee9-405f-9a50-69602c801cea admin admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:34.859180 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b501b3-dee9-405f-9a50-69602c801cea admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:34.859637 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b501b3-dee9-405f-9a50-69602c801cea admin admin] PATH_INFO: `/v3/projects` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:34.860481 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b501b3-dee9-405f-9a50-69602c801cea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:34.862902 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b501b3-dee9-405f-9a50-69602c801cea admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:34.870340 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b68e16-b770-4084-96cc-55ff3867325f admin admin] RBAC: Authorizing `identity:create_grant(project_id=2dfa1d5e59664af5a450bb1003ffbb4d, user_id=d15970cea431479f8b0aec00863efa48, role_id=7b549d58d1724395b343609d895c80d0)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:34.875023 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b68e16-b770-4084-96cc-55ff3867325f admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:34.896223 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 929/1870] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jul 14 09:18:34 2022] PUT /identity/v3/projects/2dfa1d5e59664af5a450bb1003ffbb4d/users/d15970cea431479f8b0aec00863efa48/roles/7b549d58d1724395b343609d895c80d0 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:18:34.897760 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce1a3ed-bb8f-49b3-af7f-d6a8bdd00511 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:34.898423 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce1a3ed-bb8f-49b3-af7f-d6a8bdd00511 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:34.899019 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce1a3ed-bb8f-49b3-af7f-d6a8bdd00511 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:34.903325 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 942/1871] 10.222.0.26 () {60 vars in 1152 bytes} [Thu Jul 14 09:18:34 2022] POST /identity/v3/projects => generated 367 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 09:18:34.903923 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b501b3-dee9-405f-9a50-69602c801cea admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:34.938690 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-1ce1a3ed-bb8f-49b3-af7f-d6a8bdd00511 None None] MFA Rules not processed for user `b4eb9317307e4ebebe4d430ef757bb46`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:18:34.941115 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-1ce1a3ed-bb8f-49b3-af7f-d6a8bdd00511 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:18:34.995977 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b501b3-dee9-405f-9a50-69602c801cea admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:35.007594 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30392354-32f3-4222-89dd-6d6a73d0b185 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:35.008297 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30392354-32f3-4222-89dd-6d6a73d0b185 admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:35.008579 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30392354-32f3-4222-89dd-6d6a73d0b185 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:35.008855 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30392354-32f3-4222-89dd-6d6a73d0b185 admin admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:35.009601 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30392354-32f3-4222-89dd-6d6a73d0b185 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:35.013351 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30392354-32f3-4222-89dd-6d6a73d0b185 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:35.037262 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 943/1872] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jul 14 09:18:34 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 09:18:35.040789 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30392354-32f3-4222-89dd-6d6a73d0b185 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:35.059861 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30392354-32f3-4222-89dd-6d6a73d0b185 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:35.061889 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 930/1873] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:18:34 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:18:35.066479 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce1a3ed-bb8f-49b3-af7f-d6a8bdd00511 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:35.080543 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb39c2a0-4997-49c9-898f-7975ab044d42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:35.081242 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb39c2a0-4997-49c9-898f-7975ab044d42 admin admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:35.081524 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb39c2a0-4997-49c9-898f-7975ab044d42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:35.081799 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb39c2a0-4997-49c9-898f-7975ab044d42 admin admin] PATH_INFO: `/v3/users` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:35.082502 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb39c2a0-4997-49c9-898f-7975ab044d42 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:35.084511 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb39c2a0-4997-49c9-898f-7975ab044d42 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:35.086781 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce1a3ed-bb8f-49b3-af7f-d6a8bdd00511 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:35.098286 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399d046f-6855-4cbe-b5a8-d81053609cd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:35.099106 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399d046f-6855-4cbe-b5a8-d81053609cd2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:35.099489 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399d046f-6855-4cbe-b5a8-d81053609cd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:35.099763 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399d046f-6855-4cbe-b5a8-d81053609cd2 admin admin] PATH_INFO: `/v3/projects/2dfa1d5e59664af5a450bb1003ffbb4d/users/d15970cea431479f8b0aec00863efa48/roles/7b549d58d1724395b343609d895c80d0` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:35.103410 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399d046f-6855-4cbe-b5a8-d81053609cd2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2dfa1d5e59664af5a450bb1003ffbb4d, user_id=d15970cea431479f8b0aec00863efa48, role_id=7b549d58d1724395b343609d895c80d0)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:35.106142 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399d046f-6855-4cbe-b5a8-d81053609cd2 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:35.126753 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 944/1874] 10.222.0.26 () {60 vars in 1143 bytes} [Thu Jul 14 09:18:35 2022] POST /identity/v3/users => generated 290 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 09:18:35.126753 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb39c2a0-4997-49c9-898f-7975ab044d42 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:35.129359 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-cb39c2a0-4997-49c9-898f-7975ab044d42 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:18:35.140174 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 931/1875] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jul 14 09:18:35 2022] PUT /identity/v3/projects/2dfa1d5e59664af5a450bb1003ffbb4d/users/d15970cea431479f8b0aec00863efa48/roles/7b549d58d1724395b343609d895c80d0 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:18:35.140174 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399d046f-6855-4cbe-b5a8-d81053609cd2 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:35.162368 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:18:35.281298 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399d046f-6855-4cbe-b5a8-d81053609cd2 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:35.289080 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812f6f0a-c579-4c03-9cc3-317202fe36cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:35.290765 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb39c2a0-4997-49c9-898f-7975ab044d42 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:35.291797 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812f6f0a-c579-4c03-9cc3-317202fe36cc admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:35.292188 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812f6f0a-c579-4c03-9cc3-317202fe36cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:35.292627 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812f6f0a-c579-4c03-9cc3-317202fe36cc admin admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:35.293619 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812f6f0a-c579-4c03-9cc3-317202fe36cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:35.299532 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812f6f0a-c579-4c03-9cc3-317202fe36cc admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:35.327263 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 932/1876] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jul 14 09:18:35 2022] GET /identity/v3/roles => generated 1420 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 09:18:35.333692 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e32dff4-8cda-4726-8b63-9eaed255b076 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:35.334004 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e32dff4-8cda-4726-8b63-9eaed255b076 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:35.334295 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e32dff4-8cda-4726-8b63-9eaed255b076 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:35.377581 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1b9d44-b2ac-4482-b81f-7dde8d9ee8e5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e76f1952a5b146808844b9f2b7395035', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:35.378307 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1b9d44-b2ac-4482-b81f-7dde8d9ee8e5 service cinder] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:35.378661 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1b9d44-b2ac-4482-b81f-7dde8d9ee8e5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:35.379139 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1b9d44-b2ac-4482-b81f-7dde8d9ee8e5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:35.379598 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-7e32dff4-8cda-4726-8b63-9eaed255b076 None None] MFA Rules not processed for user `d15970cea431479f8b0aec00863efa48`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:18:35.381138 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-7e32dff4-8cda-4726-8b63-9eaed255b076 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:18:35.421394 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:18:35.495362 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 933/1877] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:18:35 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:18:35.496259 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e32dff4-8cda-4726-8b63-9eaed255b076 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:35.501617 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1b9d44-b2ac-4482-b81f-7dde8d9ee8e5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:35.503072 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1b9d44-b2ac-4482-b81f-7dde8d9ee8e5 service cinder] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:35.507235 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e32dff4-8cda-4726-8b63-9eaed255b076 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:35.512130 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106dffd1-d3dc-41ca-a4bf-f7f5785cbb0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:35.512864 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 945/1878] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:18:35 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:18:35.513411 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106dffd1-d3dc-41ca-a4bf-f7f5785cbb0d admin admin] REQUEST_METHOD: `PUT` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:35.513667 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106dffd1-d3dc-41ca-a4bf-f7f5785cbb0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:35.513919 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106dffd1-d3dc-41ca-a4bf-f7f5785cbb0d admin admin] PATH_INFO: `/v3/projects/44fd8600fda943249b00cccb59f24d0f/users/f0610972ae3d4c75be10869d1063f5fd/roles/7b549d58d1724395b343609d895c80d0` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:35.537995 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106dffd1-d3dc-41ca-a4bf-f7f5785cbb0d admin admin] RBAC: Authorizing `identity:create_grant(project_id=44fd8600fda943249b00cccb59f24d0f, user_id=f0610972ae3d4c75be10869d1063f5fd, role_id=7b549d58d1724395b343609d895c80d0)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:35.541008 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106dffd1-d3dc-41ca-a4bf-f7f5785cbb0d admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:35.556073 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1b9d44-b2ac-4482-b81f-7dde8d9ee8e5 service cinder] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:35.567807 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 934/1879] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jul 14 09:18:35 2022] PUT /identity/v3/projects/44fd8600fda943249b00cccb59f24d0f/users/f0610972ae3d4c75be10869d1063f5fd/roles/7b549d58d1724395b343609d895c80d0 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:18:35.572120 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106dffd1-d3dc-41ca-a4bf-f7f5785cbb0d admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:35.577095 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1b9d44-b2ac-4482-b81f-7dde8d9ee8e5 service cinder] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:35.663841 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106dffd1-d3dc-41ca-a4bf-f7f5785cbb0d admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:35.667867 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5c0773-96a0-4643-b5ab-8096b92461e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:35.668828 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5c0773-96a0-4643-b5ab-8096b92461e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:35.669153 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5c0773-96a0-4643-b5ab-8096b92461e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:35.669468 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5c0773-96a0-4643-b5ab-8096b92461e5 admin admin] PATH_INFO: `/v3/users/b4eb9317307e4ebebe4d430ef757bb46` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:35.670041 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86a69d4-930e-4bc4-b104-6335897097bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:35.670863 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86a69d4-930e-4bc4-b104-6335897097bc admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:35.671071 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5c0773-96a0-4643-b5ab-8096b92461e5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b4eb9317307e4ebebe4d430ef757bb46)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:35.671370 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86a69d4-930e-4bc4-b104-6335897097bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:35.671873 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86a69d4-930e-4bc4-b104-6335897097bc admin admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:35.672554 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86a69d4-930e-4bc4-b104-6335897097bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:35.673818 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5c0773-96a0-4643-b5ab-8096b92461e5 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:35.674352 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86a69d4-930e-4bc4-b104-6335897097bc admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:35.692024 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 935/1880] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jul 14 09:18:35 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 09:18:35.692024 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86a69d4-930e-4bc4-b104-6335897097bc admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:35.705511 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:18:35.734898 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-bf5c0773-96a0-4643-b5ab-8096b92461e5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b4eb9317307e4ebebe4d430ef757bb46'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:35.735854 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-bf5c0773-96a0-4643-b5ab-8096b92461e5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b4eb9317307e4ebebe4d430ef757bb46'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:35.756043 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-bf5c0773-96a0-4643-b5ab-8096b92461e5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b4eb9317307e4ebebe4d430ef757bb46'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:35.761938 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86a69d4-930e-4bc4-b104-6335897097bc admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:35.766076 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44e4a20-a327-47af-9fbb-cb62d2089689 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e76f1952a5b146808844b9f2b7395035', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:35.766216 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-bf5c0773-96a0-4643-b5ab-8096b92461e5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b4eb9317307e4ebebe4d430ef757bb46'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:35.766862 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44e4a20-a327-47af-9fbb-cb62d2089689 service cinder] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:35.766988 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-bf5c0773-96a0-4643-b5ab-8096b92461e5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b4eb9317307e4ebebe4d430ef757bb46'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:35.767353 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44e4a20-a327-47af-9fbb-cb62d2089689 service cinder] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:35.767726 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44e4a20-a327-47af-9fbb-cb62d2089689 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:35.770075 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-f44e4a20-a327-47af-9fbb-cb62d2089689 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:18:35.795805 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 946/1881] 10.222.0.26 () {58 vars in 1223 bytes} [Thu Jul 14 09:18:35 2022] DELETE /identity/v3/users/b4eb9317307e4ebebe4d430ef757bb46 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 09:18:35.799555 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5c0773-96a0-4643-b5ab-8096b92461e5 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:35.801900 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-bf5c0773-96a0-4643-b5ab-8096b92461e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:18:35.845814 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:18:35.884346 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44e4a20-a327-47af-9fbb-cb62d2089689 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:35.884346 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44e4a20-a327-47af-9fbb-cb62d2089689 service cinder] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:35.896338 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 936/1882] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:18:35 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:18:35.898257 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb61277-8da3-4d7a-9d22-d4a131708b15 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:35.899173 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb61277-8da3-4d7a-9d22-d4a131708b15 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:35.899617 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb61277-8da3-4d7a-9d22-d4a131708b15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:35.930520 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-bfb61277-8da3-4d7a-9d22-d4a131708b15 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:18:35.931791 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-bfb61277-8da3-4d7a-9d22-d4a131708b15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:18:35.942239 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5c0773-96a0-4643-b5ab-8096b92461e5 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:35.943608 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 937/1883] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:18:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:18:35.956861 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacb4d05-ba2e-4d8d-bb5c-0bde599c644c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:35.957168 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb61277-8da3-4d7a-9d22-d4a131708b15 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:35.958165 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacb4d05-ba2e-4d8d-bb5c-0bde599c644c admin admin] REQUEST_METHOD: `PUT` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:35.958565 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacb4d05-ba2e-4d8d-bb5c-0bde599c644c admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:35.958932 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacb4d05-ba2e-4d8d-bb5c-0bde599c644c admin admin] PATH_INFO: `/v3/projects/44fd8600fda943249b00cccb59f24d0f/users/f0610972ae3d4c75be10869d1063f5fd/roles/7b549d58d1724395b343609d895c80d0` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:35.959301 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-bfb61277-8da3-4d7a-9d22-d4a131708b15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:18:35.963347 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacb4d05-ba2e-4d8d-bb5c-0bde599c644c admin admin] RBAC: Authorizing `identity:create_grant(project_id=44fd8600fda943249b00cccb59f24d0f, user_id=f0610972ae3d4c75be10869d1063f5fd, role_id=7b549d58d1724395b343609d895c80d0)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:35.967243 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacb4d05-ba2e-4d8d-bb5c-0bde599c644c admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:35.999344 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 947/1884] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jul 14 09:18:35 2022] PUT /identity/v3/projects/44fd8600fda943249b00cccb59f24d0f/users/f0610972ae3d4c75be10869d1063f5fd/roles/7b549d58d1724395b343609d895c80d0 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:18:36.000637 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacb4d05-ba2e-4d8d-bb5c-0bde599c644c admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:36.003016 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-bacb4d05-ba2e-4d8d-bb5c-0bde599c644c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:18:36.053551 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb61277-8da3-4d7a-9d22-d4a131708b15 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:36.112598 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacb4d05-ba2e-4d8d-bb5c-0bde599c644c admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:36.116407 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4e3c4f-beef-4e57-9268-4b2232deb4b9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e76f1952a5b146808844b9f2b7395035', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:36.117067 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4e3c4f-beef-4e57-9268-4b2232deb4b9 service cinder] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:36.117434 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4e3c4f-beef-4e57-9268-4b2232deb4b9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:36.117777 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4e3c4f-beef-4e57-9268-4b2232deb4b9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:36.119896 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-1d4e3c4f-beef-4e57-9268-4b2232deb4b9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:18:36.142153 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef1b47d-6ff8-4f77-8093-90d7cc00996c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '952911c492444ffea4b211a24b564117', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:36.143654 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef1b47d-6ff8-4f77-8093-90d7cc00996c service neutron] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:36.144086 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef1b47d-6ff8-4f77-8093-90d7cc00996c service neutron] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:36.145260 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef1b47d-6ff8-4f77-8093-90d7cc00996c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:36.180993 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:DELETE=7,keystone:SELECT=91,keystone:INSERT=2 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:18:36.206580 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4e3c4f-beef-4e57-9268-4b2232deb4b9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:36.208648 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4e3c4f-beef-4e57-9268-4b2232deb4b9 service cinder] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:36.235719 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 948/1885] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:18:35 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:18:36.238759 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488efdee-74af-42da-bccf-feab6d20c38f None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:36.239119 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488efdee-74af-42da-bccf-feab6d20c38f None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:36.239420 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488efdee-74af-42da-bccf-feab6d20c38f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:36.261556 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef1b47d-6ff8-4f77-8093-90d7cc00996c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:36.263856 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef1b47d-6ff8-4f77-8093-90d7cc00996c service neutron] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:36.281621 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 938/1886] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:18:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:18:36.302119 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-488efdee-74af-42da-bccf-feab6d20c38f None None] MFA Rules not processed for user `f0610972ae3d4c75be10869d1063f5fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:18:36.303381 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-488efdee-74af-42da-bccf-feab6d20c38f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:18:36.378524 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef1b47d-6ff8-4f77-8093-90d7cc00996c service neutron] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:36.380246 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-fef1b47d-6ff8-4f77-8093-90d7cc00996c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:18:36.404765 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef1b47d-6ff8-4f77-8093-90d7cc00996c service neutron] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:36.412159 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4013c1-2ff5-4995-ad1c-03a143a200e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:36.413470 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4013c1-2ff5-4995-ad1c-03a143a200e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:36.414789 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4013c1-2ff5-4995-ad1c-03a143a200e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:36.415254 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4013c1-2ff5-4995-ad1c-03a143a200e3 admin admin] PATH_INFO: `/v3/users/436ff0103dcf401d9363b2accfe635cb` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:36.417641 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4013c1-2ff5-4995-ad1c-03a143a200e3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=436ff0103dcf401d9363b2accfe635cb)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:36.421639 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4013c1-2ff5-4995-ad1c-03a143a200e3 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:36.426157 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 949/1887] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:18:36 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:18:36.439626 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488efdee-74af-42da-bccf-feab6d20c38f None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:36.442633 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-488efdee-74af-42da-bccf-feab6d20c38f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:18:36.473840 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488efdee-74af-42da-bccf-feab6d20c38f None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:36.493209 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f364e015-cf9b-4fe7-b132-69461064557b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e76f1952a5b146808844b9f2b7395035', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:36.493872 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f364e015-cf9b-4fe7-b132-69461064557b service cinder] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:36.494156 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f364e015-cf9b-4fe7-b132-69461064557b service cinder] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:36.494555 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f364e015-cf9b-4fe7-b132-69461064557b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:36.515769 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f364e015-cf9b-4fe7-b132-69461064557b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:36.517265 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f364e015-cf9b-4fe7-b132-69461064557b service cinder] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:36.530112 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:18:36.553847 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-fd4013c1-2ff5-4995-ad1c-03a143a200e3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '436ff0103dcf401d9363b2accfe635cb'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:36.557865 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 950/1888] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:18:36 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:18:36.571945 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-fd4013c1-2ff5-4995-ad1c-03a143a200e3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '436ff0103dcf401d9363b2accfe635cb'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:36.573106 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-fd4013c1-2ff5-4995-ad1c-03a143a200e3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '436ff0103dcf401d9363b2accfe635cb'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:36.574025 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-fd4013c1-2ff5-4995-ad1c-03a143a200e3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '436ff0103dcf401d9363b2accfe635cb'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:36.589847 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-fd4013c1-2ff5-4995-ad1c-03a143a200e3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '436ff0103dcf401d9363b2accfe635cb'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:36.617224 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 939/1889] 10.222.0.26 () {58 vars in 1223 bytes} [Thu Jul 14 09:18:36 2022] DELETE /identity/v3/users/436ff0103dcf401d9363b2accfe635cb => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:18:36.621840 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43aad7df-24af-4067-b61c-4ef84c944704 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:36.622129 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43aad7df-24af-4067-b61c-4ef84c944704 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:36.622407 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43aad7df-24af-4067-b61c-4ef84c944704 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:36.649911 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-43aad7df-24af-4067-b61c-4ef84c944704 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:18:36.651125 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-43aad7df-24af-4067-b61c-4ef84c944704 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:18:36.705661 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4013c1-2ff5-4995-ad1c-03a143a200e3 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:36.705767 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-fd4013c1-2ff5-4995-ad1c-03a143a200e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:18:36.742362 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 951/1890] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:18:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 09:18:36.827314 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43aad7df-24af-4067-b61c-4ef84c944704 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:36.830971 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-43aad7df-24af-4067-b61c-4ef84c944704 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:18:36.890938 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4013c1-2ff5-4995-ad1c-03a143a200e3 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:36.901015 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442eba3d-d517-487f-a89c-2e5767104023 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e76f1952a5b146808844b9f2b7395035', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:36.902508 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442eba3d-d517-487f-a89c-2e5767104023 service cinder] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:36.902823 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442eba3d-d517-487f-a89c-2e5767104023 service cinder] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:36.904277 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442eba3d-d517-487f-a89c-2e5767104023 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:36.908623 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-442eba3d-d517-487f-a89c-2e5767104023 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:18:36.951722 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43aad7df-24af-4067-b61c-4ef84c944704 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:36.973782 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ea13c1-bf50-40d4-810c-c6adab5083be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '952911c492444ffea4b211a24b564117', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:36.974465 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ea13c1-bf50-40d4-810c-c6adab5083be service neutron] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:36.974784 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ea13c1-bf50-40d4-810c-c6adab5083be service neutron] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:36.975063 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ea13c1-bf50-40d4-810c-c6adab5083be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:36.986963 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ea13c1-bf50-40d4-810c-c6adab5083be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:36.988816 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ea13c1-bf50-40d4-810c-c6adab5083be service neutron] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:36.999874 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 952/1891] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:18:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:18:37.012872 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442eba3d-d517-487f-a89c-2e5767104023 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:37.014537 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442eba3d-d517-487f-a89c-2e5767104023 service cinder] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:37.026137 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 940/1892] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:18:36 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 09:18:37.073610 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ea13c1-bf50-40d4-810c-c6adab5083be service neutron] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:37.075481 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-e2ea13c1-bf50-40d4-810c-c6adab5083be service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:18:37.084184 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:18:37.099177 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ea13c1-bf50-40d4-810c-c6adab5083be service neutron] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:37.104563 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe0e604-6f4f-4576-af2b-30960aa85517 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:37.105445 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe0e604-6f4f-4576-af2b-30960aa85517 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:37.105723 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe0e604-6f4f-4576-af2b-30960aa85517 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:37.106116 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe0e604-6f4f-4576-af2b-30960aa85517 admin admin] PATH_INFO: `/v3/projects/9e608162ad754d92b7dbcb9f7eb99368` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:37.107368 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe0e604-6f4f-4576-af2b-30960aa85517 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9e608162ad754d92b7dbcb9f7eb99368)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:37.109548 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe0e604-6f4f-4576-af2b-30960aa85517 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:37.215545 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-5fe0e604-6f4f-4576-af2b-30960aa85517 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9e608162ad754d92b7dbcb9f7eb99368', 'request_id': 'req-5fe0e604-6f4f-4576-af2b-30960aa85517'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:37.235891 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-5fe0e604-6f4f-4576-af2b-30960aa85517 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9e608162ad754d92b7dbcb9f7eb99368', 'request_id': 'req-5fe0e604-6f4f-4576-af2b-30960aa85517'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:37.250350 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-5fe0e604-6f4f-4576-af2b-30960aa85517 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9e608162ad754d92b7dbcb9f7eb99368', 'request_id': 'req-5fe0e604-6f4f-4576-af2b-30960aa85517'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:37.291480 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-5fe0e604-6f4f-4576-af2b-30960aa85517 admin admin] The token cache is being invalidate because project 9e608162ad754d92b7dbcb9f7eb99368 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80382) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 09:18:37.291807 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-5fe0e604-6f4f-4576-af2b-30960aa85517 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:37.292881 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-5fe0e604-6f4f-4576-af2b-30960aa85517 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:37.299366 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 953/1893] 10.222.0.26 () {58 vars in 1232 bytes} [Thu Jul 14 09:18:37 2022] DELETE /identity/v3/projects/9e608162ad754d92b7dbcb9f7eb99368 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:18:37.302186 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0692d4a-7bc2-4b68-b450-6f28262410ca None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:37.302491 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0692d4a-7bc2-4b68-b450-6f28262410ca None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:37.302808 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0692d4a-7bc2-4b68-b450-6f28262410ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:37.364865 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-e0692d4a-7bc2-4b68-b450-6f28262410ca None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:18:37.368221 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-e0692d4a-7bc2-4b68-b450-6f28262410ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:18:37.521317 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 941/1894] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:18:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:18:37.531110 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcc79e4-39f6-4627-a0e1-444a4a07daa3 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:37.531110 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcc79e4-39f6-4627-a0e1-444a4a07daa3 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:37.531751 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcc79e4-39f6-4627-a0e1-444a4a07daa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:37.564830 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-0dcc79e4-39f6-4627-a0e1-444a4a07daa3 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:18:37.566093 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-0dcc79e4-39f6-4627-a0e1-444a4a07daa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:18:37.597128 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 954/1895] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:18:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:18:37.598346 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0692d4a-7bc2-4b68-b450-6f28262410ca None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:37.623914 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0692d4a-7bc2-4b68-b450-6f28262410ca None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:37.632571 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526916d3-c5af-49a7-bb78-45958596fa95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:37.632808 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526916d3-c5af-49a7-bb78-45958596fa95 admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:37.633051 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526916d3-c5af-49a7-bb78-45958596fa95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:37.633300 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526916d3-c5af-49a7-bb78-45958596fa95 admin admin] PATH_INFO: `/v3/domains` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:37.634028 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526916d3-c5af-49a7-bb78-45958596fa95 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:18:37.634344 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526916d3-c5af-49a7-bb78-45958596fa95 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:37.636177 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526916d3-c5af-49a7-bb78-45958596fa95 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:37.656997 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:18:37.674762 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 942/1896] 10.222.0.26 () {58 vars in 1152 bytes} [Thu Jul 14 09:18:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 09:18:37.675906 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcc79e4-39f6-4627-a0e1-444a4a07daa3 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:37.695989 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcc79e4-39f6-4627-a0e1-444a4a07daa3 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:37.701999 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d27a1fb-af79-43b3-87f5-86a736a1ed3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:37.702699 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d27a1fb-af79-43b3-87f5-86a736a1ed3f admin admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:37.702984 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d27a1fb-af79-43b3-87f5-86a736a1ed3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:37.703322 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d27a1fb-af79-43b3-87f5-86a736a1ed3f admin admin] PATH_INFO: `/v3/projects` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:37.704218 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d27a1fb-af79-43b3-87f5-86a736a1ed3f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:37.706217 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d27a1fb-af79-43b3-87f5-86a736a1ed3f admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:37.749140 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 955/1897] 10.222.0.26 () {60 vars in 1152 bytes} [Thu Jul 14 09:18:37 2022] POST /identity/v3/projects => generated 375 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 09:18:37.750440 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526916d3-c5af-49a7-bb78-45958596fa95 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:37.762189 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526916d3-c5af-49a7-bb78-45958596fa95 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:37.861254 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45fce42-12eb-45d3-9abd-6246f40f991a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:37.862469 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45fce42-12eb-45d3-9abd-6246f40f991a admin admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:37.862834 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45fce42-12eb-45d3-9abd-6246f40f991a admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:37.863228 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45fce42-12eb-45d3-9abd-6246f40f991a admin admin] PATH_INFO: `/v3/users` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:37.864037 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45fce42-12eb-45d3-9abd-6246f40f991a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:37.866415 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45fce42-12eb-45d3-9abd-6246f40f991a admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:37.902897 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 943/1898] 10.222.0.26 () {60 vars in 1143 bytes} [Thu Jul 14 09:18:37 2022] POST /identity/v3/users => generated 288 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 09:18:37.903858 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d27a1fb-af79-43b3-87f5-86a736a1ed3f admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:37.917424 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45fce42-12eb-45d3-9abd-6246f40f991a admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:37.923465 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-f45fce42-12eb-45d3-9abd-6246f40f991a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:18:37.937194 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d27a1fb-af79-43b3-87f5-86a736a1ed3f admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:37.951463 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e89b82d-800e-42dd-8bb0-391cd6a213b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:37.951463 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e89b82d-800e-42dd-8bb0-391cd6a213b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:37.951463 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e89b82d-800e-42dd-8bb0-391cd6a213b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:37.951463 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e89b82d-800e-42dd-8bb0-391cd6a213b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:37.951789 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e89b82d-800e-42dd-8bb0-391cd6a213b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:37.962692 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e89b82d-800e-42dd-8bb0-391cd6a213b0 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:37.966509 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45fce42-12eb-45d3-9abd-6246f40f991a admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:37.981390 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a2b7e2-db40-47c1-a944-441482472516 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:37.982286 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a2b7e2-db40-47c1-a944-441482472516 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:37.982571 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a2b7e2-db40-47c1-a944-441482472516 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:37.982847 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a2b7e2-db40-47c1-a944-441482472516 admin admin] PATH_INFO: `/v3/projects/a49a3c1ac05e4d79822a655a88277da0` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:37.984563 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a2b7e2-db40-47c1-a944-441482472516 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a49a3c1ac05e4d79822a655a88277da0)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:37.986516 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a2b7e2-db40-47c1-a944-441482472516 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:37.988774 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 956/1899] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jul 14 09:18:37 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 09:18:37.989784 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e89b82d-800e-42dd-8bb0-391cd6a213b0 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:38.008476 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e89b82d-800e-42dd-8bb0-391cd6a213b0 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:38.021771 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ddacb8-3f1e-47c5-82a2-04383f0fd1fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:38.022555 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ddacb8-3f1e-47c5-82a2-04383f0fd1fe admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:38.022771 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ddacb8-3f1e-47c5-82a2-04383f0fd1fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:38.023054 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ddacb8-3f1e-47c5-82a2-04383f0fd1fe admin admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:38.023849 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ddacb8-3f1e-47c5-82a2-04383f0fd1fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:38.025413 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ddacb8-3f1e-47c5-82a2-04383f0fd1fe admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:38.048676 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 957/1900] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jul 14 09:18:37 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 09:18:38.051641 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ddacb8-3f1e-47c5-82a2-04383f0fd1fe admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:38.101655 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ddacb8-3f1e-47c5-82a2-04383f0fd1fe admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:38.153751 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-e1a2b7e2-db40-47c1-a944-441482472516 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a49a3c1ac05e4d79822a655a88277da0', 'request_id': 'req-e1a2b7e2-db40-47c1-a944-441482472516'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:38.186321 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-e1a2b7e2-db40-47c1-a944-441482472516 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a49a3c1ac05e4d79822a655a88277da0', 'request_id': 'req-e1a2b7e2-db40-47c1-a944-441482472516'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:38.212040 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-e1a2b7e2-db40-47c1-a944-441482472516 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a49a3c1ac05e4d79822a655a88277da0', 'request_id': 'req-e1a2b7e2-db40-47c1-a944-441482472516'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:38.237255 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-e1a2b7e2-db40-47c1-a944-441482472516 admin admin] The token cache is being invalidate because project a49a3c1ac05e4d79822a655a88277da0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80381) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 09:18:38.237553 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-e1a2b7e2-db40-47c1-a944-441482472516 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:38.241626 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-e1a2b7e2-db40-47c1-a944-441482472516 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:38.243656 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 944/1901] 10.222.0.26 () {58 vars in 1232 bytes} [Thu Jul 14 09:18:37 2022] DELETE /identity/v3/projects/a49a3c1ac05e4d79822a655a88277da0 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:18:38.248372 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a2b7e2-db40-47c1-a944-441482472516 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:38.251937 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-e1a2b7e2-db40-47c1-a944-441482472516 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:18:38.258430 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4e8ea8-5bf9-45f7-acf0-6863a39b643e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:38.259397 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4e8ea8-5bf9-45f7-acf0-6863a39b643e admin admin] REQUEST_METHOD: `PUT` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:38.259678 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4e8ea8-5bf9-45f7-acf0-6863a39b643e admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:38.260494 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4e8ea8-5bf9-45f7-acf0-6863a39b643e admin admin] PATH_INFO: `/v3/projects/d4f7361e8c32486eaf05bb15f240ba51/users/23e0aaea1ff24fdfbfe23cdfbdb5ffe3/roles/d1691322ce6a4edab7ce722edf0f2988` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:38.295831 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a2b7e2-db40-47c1-a944-441482472516 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:38.310759 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4e8ea8-5bf9-45f7-acf0-6863a39b643e admin admin] RBAC: Authorizing `identity:create_grant(project_id=d4f7361e8c32486eaf05bb15f240ba51, user_id=23e0aaea1ff24fdfbfe23cdfbdb5ffe3, role_id=d1691322ce6a4edab7ce722edf0f2988)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:38.311763 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61aa9265-9e49-46e6-ac81-6e4b47a7e60c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:38.312862 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61aa9265-9e49-46e6-ac81-6e4b47a7e60c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:38.313302 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61aa9265-9e49-46e6-ac81-6e4b47a7e60c admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:38.313724 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4e8ea8-5bf9-45f7-acf0-6863a39b643e admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:38.313859 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61aa9265-9e49-46e6-ac81-6e4b47a7e60c admin admin] PATH_INFO: `/v3/users/3f0a6ce265fd4b608ac5c264c89a9fd4` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:38.320424 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61aa9265-9e49-46e6-ac81-6e4b47a7e60c admin admin] RBAC: Authorizing `identity:delete_user(user_id=3f0a6ce265fd4b608ac5c264c89a9fd4)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:38.329596 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61aa9265-9e49-46e6-ac81-6e4b47a7e60c admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:38.333509 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 958/1902] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jul 14 09:18:38 2022] PUT /identity/v3/projects/d4f7361e8c32486eaf05bb15f240ba51/users/23e0aaea1ff24fdfbfe23cdfbdb5ffe3/roles/d1691322ce6a4edab7ce722edf0f2988 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:18:38.344488 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9a1c16-783c-46c1-830c-02b14f646ab0 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:38.344917 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9a1c16-783c-46c1-830c-02b14f646ab0 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:38.345291 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9a1c16-783c-46c1-830c-02b14f646ab0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:38.388795 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-6a9a1c16-783c-46c1-830c-02b14f646ab0 None None] MFA Rules not processed for user `23e0aaea1ff24fdfbfe23cdfbdb5ffe3`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:18:38.390370 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-6a9a1c16-783c-46c1-830c-02b14f646ab0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:18:38.401438 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:18:38.429203 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-61aa9265-9e49-46e6-ac81-6e4b47a7e60c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3f0a6ce265fd4b608ac5c264c89a9fd4'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:38.442920 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-61aa9265-9e49-46e6-ac81-6e4b47a7e60c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3f0a6ce265fd4b608ac5c264c89a9fd4'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:38.444792 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-61aa9265-9e49-46e6-ac81-6e4b47a7e60c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3f0a6ce265fd4b608ac5c264c89a9fd4'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:38.445891 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-61aa9265-9e49-46e6-ac81-6e4b47a7e60c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3f0a6ce265fd4b608ac5c264c89a9fd4'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:38.464262 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-61aa9265-9e49-46e6-ac81-6e4b47a7e60c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3f0a6ce265fd4b608ac5c264c89a9fd4'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:38.473509 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=80,keystone:INSERT=5,keystone:DELETE=15 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:18:38.527264 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 945/1903] 10.222.0.26 () {58 vars in 1223 bytes} [Thu Jul 14 09:18:38 2022] DELETE /identity/v3/users/3f0a6ce265fd4b608ac5c264c89a9fd4 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:18:38.578780 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 959/1904] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:18:38 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:18:38.580615 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61aa9265-9e49-46e6-ac81-6e4b47a7e60c admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:38.582789 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-61aa9265-9e49-46e6-ac81-6e4b47a7e60c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:18:38.716393 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61aa9265-9e49-46e6-ac81-6e4b47a7e60c admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:38.722495 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d429dde1-4aea-4b51-a2b4-e4e14943df02 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '872dbb91c9e54f13b1a31fc8242126df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:38.723492 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d429dde1-4aea-4b51-a2b4-e4e14943df02 service glance] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:38.723900 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d429dde1-4aea-4b51-a2b4-e4e14943df02 service glance] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:38.724967 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d429dde1-4aea-4b51-a2b4-e4e14943df02 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:38.864392 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d429dde1-4aea-4b51-a2b4-e4e14943df02 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:38.868119 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d429dde1-4aea-4b51-a2b4-e4e14943df02 service glance] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:38.897202 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 946/1905] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:18:38 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:18:39.159399 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9a1c16-783c-46c1-830c-02b14f646ab0 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:39.161264 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-6a9a1c16-783c-46c1-830c-02b14f646ab0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:18:39.330903 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9a1c16-783c-46c1-830c-02b14f646ab0 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:39.435913 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c02afea-5840-4dad-b26e-50a5d37bd0a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:39.436802 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c02afea-5840-4dad-b26e-50a5d37bd0a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:39.437101 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c02afea-5840-4dad-b26e-50a5d37bd0a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:39.439261 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c02afea-5840-4dad-b26e-50a5d37bd0a4 admin admin] PATH_INFO: `/v3/projects/63ced92de28b475a9ab77520514a4c37` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:39.441719 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c02afea-5840-4dad-b26e-50a5d37bd0a4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=63ced92de28b475a9ab77520514a4c37)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:39.443722 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c02afea-5840-4dad-b26e-50a5d37bd0a4 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:39.516736 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-9c02afea-5840-4dad-b26e-50a5d37bd0a4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '63ced92de28b475a9ab77520514a4c37', 'request_id': 'req-9c02afea-5840-4dad-b26e-50a5d37bd0a4'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:39.525595 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-9c02afea-5840-4dad-b26e-50a5d37bd0a4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '63ced92de28b475a9ab77520514a4c37', 'request_id': 'req-9c02afea-5840-4dad-b26e-50a5d37bd0a4'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:39.528706 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:18:39.541953 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-9c02afea-5840-4dad-b26e-50a5d37bd0a4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '63ced92de28b475a9ab77520514a4c37', 'request_id': 'req-9c02afea-5840-4dad-b26e-50a5d37bd0a4'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:39.573050 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-9c02afea-5840-4dad-b26e-50a5d37bd0a4 admin admin] The token cache is being invalidate because project 63ced92de28b475a9ab77520514a4c37 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80382) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 09:18:39.573408 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-9c02afea-5840-4dad-b26e-50a5d37bd0a4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:39.574610 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-9c02afea-5840-4dad-b26e-50a5d37bd0a4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:39.577549 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 960/1906] 10.222.0.26 () {58 vars in 1232 bytes} [Thu Jul 14 09:18:39 2022] DELETE /identity/v3/projects/63ced92de28b475a9ab77520514a4c37 => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:18:39.590958 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84e7b7f-8804-4602-871b-0e46a510b341 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:39.591344 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84e7b7f-8804-4602-871b-0e46a510b341 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:39.591690 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84e7b7f-8804-4602-871b-0e46a510b341 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:39.624163 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-b84e7b7f-8804-4602-871b-0e46a510b341 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:18:39.625665 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-b84e7b7f-8804-4602-871b-0e46a510b341 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:18:39.801108 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 947/1907] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:18:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:18:39.808295 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15329541-f86a-4de8-a902-19cd315982a8 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:39.808769 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15329541-f86a-4de8-a902-19cd315982a8 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:39.808867 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15329541-f86a-4de8-a902-19cd315982a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:39.876008 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-15329541-f86a-4de8-a902-19cd315982a8 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:18:39.877957 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-15329541-f86a-4de8-a902-19cd315982a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:18:39.890778 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 961/1908] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:18:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 09:18:39.895440 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84e7b7f-8804-4602-871b-0e46a510b341 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:39.911763 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84e7b7f-8804-4602-871b-0e46a510b341 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:39.922867 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4262d1e0-e7d5-4208-b53a-b8d3eaaaac9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:39.923964 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4262d1e0-e7d5-4208-b53a-b8d3eaaaac9f admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:39.924574 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4262d1e0-e7d5-4208-b53a-b8d3eaaaac9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:39.924966 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4262d1e0-e7d5-4208-b53a-b8d3eaaaac9f admin admin] PATH_INFO: `/v3/domains` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:39.925880 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4262d1e0-e7d5-4208-b53a-b8d3eaaaac9f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:18:39.926495 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4262d1e0-e7d5-4208-b53a-b8d3eaaaac9f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:39.928666 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4262d1e0-e7d5-4208-b53a-b8d3eaaaac9f admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:39.953749 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 948/1909] 10.222.0.26 () {58 vars in 1152 bytes} [Thu Jul 14 09:18:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:18:39.954799 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15329541-f86a-4de8-a902-19cd315982a8 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:39.971443 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15329541-f86a-4de8-a902-19cd315982a8 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:39.979631 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88a909f-1569-404e-8a12-0b239bbde854 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:39.980427 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88a909f-1569-404e-8a12-0b239bbde854 admin admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:39.980770 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88a909f-1569-404e-8a12-0b239bbde854 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:39.981297 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88a909f-1569-404e-8a12-0b239bbde854 admin admin] PATH_INFO: `/v3/projects` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:39.982174 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88a909f-1569-404e-8a12-0b239bbde854 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:39.984325 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88a909f-1569-404e-8a12-0b239bbde854 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:40.023065 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 962/1910] 10.222.0.26 () {60 vars in 1152 bytes} [Thu Jul 14 09:18:39 2022] POST /identity/v3/projects => generated 379 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 09:18:40.027351 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4262d1e0-e7d5-4208-b53a-b8d3eaaaac9f admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:40.034399 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:18:40.038218 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4262d1e0-e7d5-4208-b53a-b8d3eaaaac9f admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:40.104259 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a16cc1d-3700-4a08-a7bc-a6f4ab60be3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:40.105019 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a16cc1d-3700-4a08-a7bc-a6f4ab60be3b admin admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:40.105390 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a16cc1d-3700-4a08-a7bc-a6f4ab60be3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:40.105731 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a16cc1d-3700-4a08-a7bc-a6f4ab60be3b admin admin] PATH_INFO: `/v3/users` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:40.106570 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a16cc1d-3700-4a08-a7bc-a6f4ab60be3b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:40.108603 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a16cc1d-3700-4a08-a7bc-a6f4ab60be3b admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:40.141567 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 949/1911] 10.222.0.26 () {60 vars in 1143 bytes} [Thu Jul 14 09:18:40 2022] POST /identity/v3/users => generated 290 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 09:18:40.142808 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88a909f-1569-404e-8a12-0b239bbde854 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:40.154512 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88a909f-1569-404e-8a12-0b239bbde854 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:40.161149 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4eb2c2-4baa-4c06-894e-71b5b1298496 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:40.162410 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4eb2c2-4baa-4c06-894e-71b5b1298496 admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:40.163261 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4eb2c2-4baa-4c06-894e-71b5b1298496 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:40.164201 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4eb2c2-4baa-4c06-894e-71b5b1298496 admin admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:40.165074 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4eb2c2-4baa-4c06-894e-71b5b1298496 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:40.167091 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4eb2c2-4baa-4c06-894e-71b5b1298496 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:40.185323 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 963/1912] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jul 14 09:18:40 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 09:18:40.191432 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a16cc1d-3700-4a08-a7bc-a6f4ab60be3b admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:40.205043 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a16cc1d-3700-4a08-a7bc-a6f4ab60be3b admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:40.212335 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0e9b3e-85d4-49a0-9682-f30288f1c09e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:40.213164 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0e9b3e-85d4-49a0-9682-f30288f1c09e admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:40.213877 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0e9b3e-85d4-49a0-9682-f30288f1c09e admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:40.214181 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0e9b3e-85d4-49a0-9682-f30288f1c09e admin admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:40.215069 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0e9b3e-85d4-49a0-9682-f30288f1c09e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:40.216888 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0e9b3e-85d4-49a0-9682-f30288f1c09e admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:40.230492 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 950/1913] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jul 14 09:18:40 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 09:18:40.237979 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4eb2c2-4baa-4c06-894e-71b5b1298496 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:40.247717 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4eb2c2-4baa-4c06-894e-71b5b1298496 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:40.253662 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eae2445-61d8-4a0b-aaad-3f09bc215a48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:40.254561 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eae2445-61d8-4a0b-aaad-3f09bc215a48 admin admin] REQUEST_METHOD: `PUT` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:40.254842 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eae2445-61d8-4a0b-aaad-3f09bc215a48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:40.255111 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eae2445-61d8-4a0b-aaad-3f09bc215a48 admin admin] PATH_INFO: `/v3/projects/bc6ba239599f4512a6454e3c30d2a307/users/7305968fdeca45499a983a343ea2751c/roles/d1691322ce6a4edab7ce722edf0f2988` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:40.274428 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eae2445-61d8-4a0b-aaad-3f09bc215a48 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc6ba239599f4512a6454e3c30d2a307, user_id=7305968fdeca45499a983a343ea2751c, role_id=d1691322ce6a4edab7ce722edf0f2988)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:40.280949 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eae2445-61d8-4a0b-aaad-3f09bc215a48 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:40.301674 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 964/1914] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jul 14 09:18:40 2022] PUT /identity/v3/projects/bc6ba239599f4512a6454e3c30d2a307/users/7305968fdeca45499a983a343ea2751c/roles/d1691322ce6a4edab7ce722edf0f2988 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 09:18:40.308978 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519462c9-208d-4a3f-8b2f-a54b8e7dba64 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:40.309373 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519462c9-208d-4a3f-8b2f-a54b8e7dba64 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:40.309649 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519462c9-208d-4a3f-8b2f-a54b8e7dba64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:40.354390 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-519462c9-208d-4a3f-8b2f-a54b8e7dba64 None None] MFA Rules not processed for user `7305968fdeca45499a983a343ea2751c`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:18:40.356157 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-519462c9-208d-4a3f-8b2f-a54b8e7dba64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:18:40.492586 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 951/1915] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:18:40 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 09:18:40.497700 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eae2445-61d8-4a0b-aaad-3f09bc215a48 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:40.621428 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eae2445-61d8-4a0b-aaad-3f09bc215a48 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:40.628461 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fb6b6c-70fb-4536-942c-df53362499dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:40.629142 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fb6b6c-70fb-4536-942c-df53362499dc admin admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:40.630268 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fb6b6c-70fb-4536-942c-df53362499dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:40.631013 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fb6b6c-70fb-4536-942c-df53362499dc admin admin] PATH_INFO: `/v3/projects` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:40.633205 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fb6b6c-70fb-4536-942c-df53362499dc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:40.636835 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fb6b6c-70fb-4536-942c-df53362499dc admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:40.689048 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 965/1916] 10.222.0.26 () {60 vars in 1152 bytes} [Thu Jul 14 09:18:40 2022] POST /identity/v3/projects => generated 381 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 09:18:40.693150 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519462c9-208d-4a3f-8b2f-a54b8e7dba64 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:40.711077 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519462c9-208d-4a3f-8b2f-a54b8e7dba64 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:40.851812 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a72447-4ba9-4fc1-9758-63bf55b112ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:40.852932 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a72447-4ba9-4fc1-9758-63bf55b112ec admin admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:40.853364 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a72447-4ba9-4fc1-9758-63bf55b112ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:40.853733 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a72447-4ba9-4fc1-9758-63bf55b112ec admin admin] PATH_INFO: `/v3/users` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:40.854652 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a72447-4ba9-4fc1-9758-63bf55b112ec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:40.856899 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a72447-4ba9-4fc1-9758-63bf55b112ec admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:40.900774 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 952/1917] 10.222.0.26 () {60 vars in 1143 bytes} [Thu Jul 14 09:18:40 2022] POST /identity/v3/users => generated 297 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 09:18:40.902065 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fb6b6c-70fb-4536-942c-df53362499dc admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:40.915760 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fb6b6c-70fb-4536-942c-df53362499dc admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:40.930105 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23e1e91-9961-45a8-bc60-1c9b7ac1a043 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:40.930813 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23e1e91-9961-45a8-bc60-1c9b7ac1a043 admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:40.931100 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23e1e91-9961-45a8-bc60-1c9b7ac1a043 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:40.931405 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23e1e91-9961-45a8-bc60-1c9b7ac1a043 admin admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:40.932150 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23e1e91-9961-45a8-bc60-1c9b7ac1a043 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:40.946278 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23e1e91-9961-45a8-bc60-1c9b7ac1a043 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:40.970537 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a72447-4ba9-4fc1-9758-63bf55b112ec admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:40.973080 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-f3a72447-4ba9-4fc1-9758-63bf55b112ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:18:40.984872 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:18:40.989358 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 966/1918] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jul 14 09:18:40 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 09:18:40.996187 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23e1e91-9961-45a8-bc60-1c9b7ac1a043 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:41.002331 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a72447-4ba9-4fc1-9758-63bf55b112ec admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:41.024889 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6e8dc3-5e0b-4685-8ddf-ccca556952aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:41.025054 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23e1e91-9961-45a8-bc60-1c9b7ac1a043 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:41.027770 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6e8dc3-5e0b-4685-8ddf-ccca556952aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:41.028971 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6e8dc3-5e0b-4685-8ddf-ccca556952aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:41.029502 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6e8dc3-5e0b-4685-8ddf-ccca556952aa admin admin] PATH_INFO: `/v3/users/df88a63188cf4ab0b7e24050a83444c5` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:41.033879 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6e8dc3-5e0b-4685-8ddf-ccca556952aa admin admin] RBAC: Authorizing `identity:delete_user(user_id=df88a63188cf4ab0b7e24050a83444c5)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:41.043076 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67587e0d-91b4-4504-9ce8-d75ef0c628a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:41.043076 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67587e0d-91b4-4504-9ce8-d75ef0c628a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:41.043076 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67587e0d-91b4-4504-9ce8-d75ef0c628a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:41.043076 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67587e0d-91b4-4504-9ce8-d75ef0c628a9 admin admin] PATH_INFO: `/v3/projects/5cad85ac7d09413b828354895dbc88dd/users/3d4ea23330a54ed5b2a6fb0f44f30a20/roles/7b549d58d1724395b343609d895c80d0` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:41.053267 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67587e0d-91b4-4504-9ce8-d75ef0c628a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5cad85ac7d09413b828354895dbc88dd, user_id=3d4ea23330a54ed5b2a6fb0f44f30a20, role_id=7b549d58d1724395b343609d895c80d0)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:41.056460 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67587e0d-91b4-4504-9ce8-d75ef0c628a9 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:41.064539 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6e8dc3-5e0b-4685-8ddf-ccca556952aa admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:41.093504 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 967/1919] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jul 14 09:18:40 2022] PUT /identity/v3/projects/5cad85ac7d09413b828354895dbc88dd/users/3d4ea23330a54ed5b2a6fb0f44f30a20/roles/7b549d58d1724395b343609d895c80d0 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:18:41.099072 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67587e0d-91b4-4504-9ce8-d75ef0c628a9 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:41.174811 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-db6e8dc3-5e0b-4685-8ddf-ccca556952aa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'df88a63188cf4ab0b7e24050a83444c5'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:41.203953 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-db6e8dc3-5e0b-4685-8ddf-ccca556952aa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'df88a63188cf4ab0b7e24050a83444c5'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:41.205876 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-db6e8dc3-5e0b-4685-8ddf-ccca556952aa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'df88a63188cf4ab0b7e24050a83444c5'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:41.206764 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=93,keystone:DELETE=2,keystone:INSERT=5 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:18:41.207032 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-db6e8dc3-5e0b-4685-8ddf-ccca556952aa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'df88a63188cf4ab0b7e24050a83444c5'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:41.216281 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-db6e8dc3-5e0b-4685-8ddf-ccca556952aa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'df88a63188cf4ab0b7e24050a83444c5'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:41.254538 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 953/1920] 10.222.0.26 () {58 vars in 1223 bytes} [Thu Jul 14 09:18:40 2022] DELETE /identity/v3/users/df88a63188cf4ab0b7e24050a83444c5 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 09:18:41.259535 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5db3e9-cefb-4442-8602-b3bc62059085 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:41.259993 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5db3e9-cefb-4442-8602-b3bc62059085 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:41.261539 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5db3e9-cefb-4442-8602-b3bc62059085 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:41.270565 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67587e0d-91b4-4504-9ce8-d75ef0c628a9 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:41.287256 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-8e5db3e9-cefb-4442-8602-b3bc62059085 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:18:41.288586 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-8e5db3e9-cefb-4442-8602-b3bc62059085 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:18:41.349271 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc884f4-0187-4a2c-a840-15969aa06494 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:41.350947 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc884f4-0187-4a2c-a840-15969aa06494 admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:41.351768 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc884f4-0187-4a2c-a840-15969aa06494 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:41.352228 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc884f4-0187-4a2c-a840-15969aa06494 admin admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:41.353511 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc884f4-0187-4a2c-a840-15969aa06494 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:41.355437 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc884f4-0187-4a2c-a840-15969aa06494 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:41.371542 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 968/1921] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jul 14 09:18:41 2022] GET /identity/v3/roles => generated 1420 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 09:18:41.377792 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc884f4-0187-4a2c-a840-15969aa06494 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:41.380648 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-7bc884f4-0187-4a2c-a840-15969aa06494 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:18:41.403284 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc884f4-0187-4a2c-a840-15969aa06494 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:41.411000 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5936058-ec2e-4222-a140-1a507c72a1d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:41.412123 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5936058-ec2e-4222-a140-1a507c72a1d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:41.412645 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5936058-ec2e-4222-a140-1a507c72a1d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:41.413012 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5936058-ec2e-4222-a140-1a507c72a1d9 admin admin] PATH_INFO: `/v3/projects/5cad85ac7d09413b828354895dbc88dd/users/3d4ea23330a54ed5b2a6fb0f44f30a20/roles/7b549d58d1724395b343609d895c80d0` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:41.413147 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 954/1922] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:18:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:18:41.417407 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5936058-ec2e-4222-a140-1a507c72a1d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5cad85ac7d09413b828354895dbc88dd, user_id=3d4ea23330a54ed5b2a6fb0f44f30a20, role_id=7b549d58d1724395b343609d895c80d0)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:41.420948 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5936058-ec2e-4222-a140-1a507c72a1d9 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:41.424466 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5db3e9-cefb-4442-8602-b3bc62059085 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:41.425906 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-8e5db3e9-cefb-4442-8602-b3bc62059085 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:18:41.441451 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 969/1923] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jul 14 09:18:41 2022] PUT /identity/v3/projects/5cad85ac7d09413b828354895dbc88dd/users/3d4ea23330a54ed5b2a6fb0f44f30a20/roles/7b549d58d1724395b343609d895c80d0 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:18:41.454657 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8990c60-b456-4bd1-9b99-48cea07f1d08 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:41.455719 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8990c60-b456-4bd1-9b99-48cea07f1d08 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:41.456401 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8990c60-b456-4bd1-9b99-48cea07f1d08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:41.516157 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-b8990c60-b456-4bd1-9b99-48cea07f1d08 None None] MFA Rules not processed for user `3d4ea23330a54ed5b2a6fb0f44f30a20`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:18:41.517624 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-b8990c60-b456-4bd1-9b99-48cea07f1d08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:18:41.540606 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5db3e9-cefb-4442-8602-b3bc62059085 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:41.601049 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:18:41.634353 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 970/1924] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:18:41 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:18:41.640652 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccd5e60-60a6-4fb3-9a20-07920a2c0402 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '952911c492444ffea4b211a24b564117', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:41.641803 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccd5e60-60a6-4fb3-9a20-07920a2c0402 service neutron] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:41.641803 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccd5e60-60a6-4fb3-9a20-07920a2c0402 service neutron] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:41.641997 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8990c60-b456-4bd1-9b99-48cea07f1d08 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:41.642093 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccd5e60-60a6-4fb3-9a20-07920a2c0402 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:41.649451 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-b8990c60-b456-4bd1-9b99-48cea07f1d08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:18:41.711465 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:18:41.717930 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8990c60-b456-4bd1-9b99-48cea07f1d08 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:41.723053 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55cf249-2ae7-44f0-9cb3-cf58a5e3ad78 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e76f1952a5b146808844b9f2b7395035', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:41.724178 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55cf249-2ae7-44f0-9cb3-cf58a5e3ad78 service cinder] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:41.724778 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55cf249-2ae7-44f0-9cb3-cf58a5e3ad78 service cinder] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:41.725441 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55cf249-2ae7-44f0-9cb3-cf58a5e3ad78 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:41.740319 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55cf249-2ae7-44f0-9cb3-cf58a5e3ad78 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:41.743308 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55cf249-2ae7-44f0-9cb3-cf58a5e3ad78 service cinder] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:41.744848 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccd5e60-60a6-4fb3-9a20-07920a2c0402 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:41.746375 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccd5e60-60a6-4fb3-9a20-07920a2c0402 service neutron] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:41.757099 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 971/1925] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:18:41 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 09:18:41.759346 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 955/1926] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:18:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 09:18:41.847493 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55cf249-2ae7-44f0-9cb3-cf58a5e3ad78 service cinder] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:41.859670 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55cf249-2ae7-44f0-9cb3-cf58a5e3ad78 service cinder] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:41.863424 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65524bdd-aee0-4514-83f3-939e0e973a35 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e76f1952a5b146808844b9f2b7395035', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:41.864100 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65524bdd-aee0-4514-83f3-939e0e973a35 service cinder] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:41.864380 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65524bdd-aee0-4514-83f3-939e0e973a35 service cinder] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:41.864651 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65524bdd-aee0-4514-83f3-939e0e973a35 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:41.866527 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-65524bdd-aee0-4514-83f3-939e0e973a35 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:18:41.945157 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65524bdd-aee0-4514-83f3-939e0e973a35 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:41.946693 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65524bdd-aee0-4514-83f3-939e0e973a35 service cinder] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:41.964049 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 972/1927] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:18:41 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:18:42.313135 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccd5e60-60a6-4fb3-9a20-07920a2c0402 service neutron] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:42.314561 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-1ccd5e60-60a6-4fb3-9a20-07920a2c0402 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:18:42.327350 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccd5e60-60a6-4fb3-9a20-07920a2c0402 service neutron] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:42.331948 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ff0752-2355-4f50-8ca0-04e755fa9c00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:42.333485 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ff0752-2355-4f50-8ca0-04e755fa9c00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:42.333809 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ff0752-2355-4f50-8ca0-04e755fa9c00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:42.334096 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ff0752-2355-4f50-8ca0-04e755fa9c00 admin admin] PATH_INFO: `/v3/users/1435cdf59c774ea3b091da8723e6cbff` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:42.335336 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ff0752-2355-4f50-8ca0-04e755fa9c00 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1435cdf59c774ea3b091da8723e6cbff)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:42.337367 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ff0752-2355-4f50-8ca0-04e755fa9c00 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:42.392333 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-46ff0752-2355-4f50-8ca0-04e755fa9c00 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1435cdf59c774ea3b091da8723e6cbff'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:42.405262 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-46ff0752-2355-4f50-8ca0-04e755fa9c00 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1435cdf59c774ea3b091da8723e6cbff'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:42.406141 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-46ff0752-2355-4f50-8ca0-04e755fa9c00 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1435cdf59c774ea3b091da8723e6cbff'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:42.406893 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-46ff0752-2355-4f50-8ca0-04e755fa9c00 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1435cdf59c774ea3b091da8723e6cbff'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:42.416869 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-46ff0752-2355-4f50-8ca0-04e755fa9c00 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1435cdf59c774ea3b091da8723e6cbff'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:42.445670 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 956/1928] 10.222.0.26 () {58 vars in 1223 bytes} [Thu Jul 14 09:18:42 2022] DELETE /identity/v3/users/1435cdf59c774ea3b091da8723e6cbff => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:18:42.450464 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f98e71-9c62-470c-a58e-8826f1c094ad None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:42.450783 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f98e71-9c62-470c-a58e-8826f1c094ad None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:42.451082 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f98e71-9c62-470c-a58e-8826f1c094ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:42.472570 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-27f98e71-9c62-470c-a58e-8826f1c094ad None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:18:42.473914 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-27f98e71-9c62-470c-a58e-8826f1c094ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:18:42.542493 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 973/1929] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:18:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 09:18:42.567014 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ff0752-2355-4f50-8ca0-04e755fa9c00 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:42.568516 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-46ff0752-2355-4f50-8ca0-04e755fa9c00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:18:42.646303 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ff0752-2355-4f50-8ca0-04e755fa9c00 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:42.650084 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f8c53b-2ce0-4c4e-a222-e02f3864e678 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '952911c492444ffea4b211a24b564117', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:42.650808 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f8c53b-2ce0-4c4e-a222-e02f3864e678 service neutron] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:42.651098 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f8c53b-2ce0-4c4e-a222-e02f3864e678 service neutron] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:42.651407 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f8c53b-2ce0-4c4e-a222-e02f3864e678 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:42.661899 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:18:42.665169 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f8c53b-2ce0-4c4e-a222-e02f3864e678 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:42.669151 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f8c53b-2ce0-4c4e-a222-e02f3864e678 service neutron] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:42.682582 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 957/1930] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:18:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:18:42.865994 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f98e71-9c62-470c-a58e-8826f1c094ad None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:42.867571 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-27f98e71-9c62-470c-a58e-8826f1c094ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:18:42.885638 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f98e71-9c62-470c-a58e-8826f1c094ad None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:42.895483 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7724d4eb-b360-4644-b764-afc92865c0bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:42.896534 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7724d4eb-b360-4644-b764-afc92865c0bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:42.897009 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7724d4eb-b360-4644-b764-afc92865c0bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:42.897431 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7724d4eb-b360-4644-b764-afc92865c0bb admin admin] PATH_INFO: `/v3/projects/c3b4991d3836498383beee96b2c6507c` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:42.899069 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7724d4eb-b360-4644-b764-afc92865c0bb admin admin] RBAC: Authorizing `identity:delete_project(project_id=c3b4991d3836498383beee96b2c6507c)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:42.902205 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7724d4eb-b360-4644-b764-afc92865c0bb admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:42.979694 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-7724d4eb-b360-4644-b764-afc92865c0bb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c3b4991d3836498383beee96b2c6507c', 'request_id': 'req-7724d4eb-b360-4644-b764-afc92865c0bb'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:42.990240 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-7724d4eb-b360-4644-b764-afc92865c0bb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c3b4991d3836498383beee96b2c6507c', 'request_id': 'req-7724d4eb-b360-4644-b764-afc92865c0bb'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:42.997917 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-7724d4eb-b360-4644-b764-afc92865c0bb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c3b4991d3836498383beee96b2c6507c', 'request_id': 'req-7724d4eb-b360-4644-b764-afc92865c0bb'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:43.019449 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-7724d4eb-b360-4644-b764-afc92865c0bb admin admin] The token cache is being invalidate because project c3b4991d3836498383beee96b2c6507c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80382) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 09:18:43.019765 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-7724d4eb-b360-4644-b764-afc92865c0bb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:43.020908 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-7724d4eb-b360-4644-b764-afc92865c0bb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:43.025917 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 974/1931] 10.222.0.26 () {58 vars in 1232 bytes} [Thu Jul 14 09:18:42 2022] DELETE /identity/v3/projects/c3b4991d3836498383beee96b2c6507c => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:18:43.027983 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f8c53b-2ce0-4c4e-a222-e02f3864e678 service neutron] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:43.047036 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f8c53b-2ce0-4c4e-a222-e02f3864e678 service neutron] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:43.166832 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7724d4eb-b360-4644-b764-afc92865c0bb admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:43.167034 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044edc91-4b12-4e8b-84be-9be007774e7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:43.167154 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-7724d4eb-b360-4644-b764-afc92865c0bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:18:43.167458 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044edc91-4b12-4e8b-84be-9be007774e7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:43.167458 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044edc91-4b12-4e8b-84be-9be007774e7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:43.167458 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044edc91-4b12-4e8b-84be-9be007774e7c admin admin] PATH_INFO: `/v3/users/d15970cea431479f8b0aec00863efa48` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:43.168430 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044edc91-4b12-4e8b-84be-9be007774e7c admin admin] RBAC: Authorizing `identity:delete_user(user_id=d15970cea431479f8b0aec00863efa48)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:43.171238 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044edc91-4b12-4e8b-84be-9be007774e7c admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:43.191369 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7724d4eb-b360-4644-b764-afc92865c0bb admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:43.212593 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848ff6b7-479e-4d10-a409-7a0fd3016400 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:43.213614 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848ff6b7-479e-4d10-a409-7a0fd3016400 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:43.213901 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848ff6b7-479e-4d10-a409-7a0fd3016400 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:43.214174 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848ff6b7-479e-4d10-a409-7a0fd3016400 admin admin] PATH_INFO: `/v3/projects/b84d87c0145a4d3dbe5435d9ce225e3e` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:43.215430 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848ff6b7-479e-4d10-a409-7a0fd3016400 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b84d87c0145a4d3dbe5435d9ce225e3e)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:43.218152 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848ff6b7-479e-4d10-a409-7a0fd3016400 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:43.232811 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:18:43.289031 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-044edc91-4b12-4e8b-84be-9be007774e7c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd15970cea431479f8b0aec00863efa48'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:43.297042 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-848ff6b7-479e-4d10-a409-7a0fd3016400 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b84d87c0145a4d3dbe5435d9ce225e3e', 'request_id': 'req-848ff6b7-479e-4d10-a409-7a0fd3016400'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:43.300643 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-044edc91-4b12-4e8b-84be-9be007774e7c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd15970cea431479f8b0aec00863efa48'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:43.301481 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-044edc91-4b12-4e8b-84be-9be007774e7c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd15970cea431479f8b0aec00863efa48'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:43.302172 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-044edc91-4b12-4e8b-84be-9be007774e7c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd15970cea431479f8b0aec00863efa48'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:43.309750 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-848ff6b7-479e-4d10-a409-7a0fd3016400 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b84d87c0145a4d3dbe5435d9ce225e3e', 'request_id': 'req-848ff6b7-479e-4d10-a409-7a0fd3016400'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:43.314612 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-044edc91-4b12-4e8b-84be-9be007774e7c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd15970cea431479f8b0aec00863efa48'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:43.317282 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-848ff6b7-479e-4d10-a409-7a0fd3016400 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b84d87c0145a4d3dbe5435d9ce225e3e', 'request_id': 'req-848ff6b7-479e-4d10-a409-7a0fd3016400'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:43.336598 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-848ff6b7-479e-4d10-a409-7a0fd3016400 admin admin] The token cache is being invalidate because project b84d87c0145a4d3dbe5435d9ce225e3e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80382) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 09:18:43.337227 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-848ff6b7-479e-4d10-a409-7a0fd3016400 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:43.338179 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-848ff6b7-479e-4d10-a409-7a0fd3016400 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:43.343504 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 958/1933] 10.222.0.26 () {58 vars in 1223 bytes} [Thu Jul 14 09:18:43 2022] DELETE /identity/v3/users/d15970cea431479f8b0aec00863efa48 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 09:18:43.347740 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044edc91-4b12-4e8b-84be-9be007774e7c admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:43.349353 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 975/1933] 10.222.0.26 () {58 vars in 1232 bytes} [Thu Jul 14 09:18:43 2022] DELETE /identity/v3/projects/b84d87c0145a4d3dbe5435d9ce225e3e => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 09:18:43.349514 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-044edc91-4b12-4e8b-84be-9be007774e7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:18:43.449712 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044edc91-4b12-4e8b-84be-9be007774e7c admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:43.455885 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a393c1cc-3668-443a-86eb-69b5c325b716 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:43.458450 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a393c1cc-3668-443a-86eb-69b5c325b716 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:43.458772 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a393c1cc-3668-443a-86eb-69b5c325b716 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:43.459078 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a393c1cc-3668-443a-86eb-69b5c325b716 admin admin] PATH_INFO: `/v3/users/f0610972ae3d4c75be10869d1063f5fd` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:43.460544 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a393c1cc-3668-443a-86eb-69b5c325b716 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f0610972ae3d4c75be10869d1063f5fd)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:43.463492 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a393c1cc-3668-443a-86eb-69b5c325b716 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:43.494412 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:18:43.568052 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-a393c1cc-3668-443a-86eb-69b5c325b716 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f0610972ae3d4c75be10869d1063f5fd'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:43.582287 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-a393c1cc-3668-443a-86eb-69b5c325b716 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f0610972ae3d4c75be10869d1063f5fd'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:43.583111 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-a393c1cc-3668-443a-86eb-69b5c325b716 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f0610972ae3d4c75be10869d1063f5fd'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:43.584069 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-a393c1cc-3668-443a-86eb-69b5c325b716 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f0610972ae3d4c75be10869d1063f5fd'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:43.594506 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-a393c1cc-3668-443a-86eb-69b5c325b716 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f0610972ae3d4c75be10869d1063f5fd'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:43.630753 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 959/1934] 10.222.0.26 () {58 vars in 1223 bytes} [Thu Jul 14 09:18:43 2022] DELETE /identity/v3/users/f0610972ae3d4c75be10869d1063f5fd => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:18:43.791863 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848ff6b7-479e-4d10-a409-7a0fd3016400 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:43.794859 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-848ff6b7-479e-4d10-a409-7a0fd3016400 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:18:43.932878 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848ff6b7-479e-4d10-a409-7a0fd3016400 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:44.008548 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e8c95b-9d07-4c9e-adf4-7cd7d708eec9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:44.010051 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e8c95b-9d07-4c9e-adf4-7cd7d708eec9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:44.010338 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e8c95b-9d07-4c9e-adf4-7cd7d708eec9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:44.010611 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e8c95b-9d07-4c9e-adf4-7cd7d708eec9 admin admin] PATH_INFO: `/v3/projects/2dfa1d5e59664af5a450bb1003ffbb4d` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:44.013808 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e8c95b-9d07-4c9e-adf4-7cd7d708eec9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2dfa1d5e59664af5a450bb1003ffbb4d)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:44.015725 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e8c95b-9d07-4c9e-adf4-7cd7d708eec9 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:44.089860 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:18:44.094287 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-85e8c95b-9d07-4c9e-adf4-7cd7d708eec9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2dfa1d5e59664af5a450bb1003ffbb4d', 'request_id': 'req-85e8c95b-9d07-4c9e-adf4-7cd7d708eec9'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:44.104697 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-85e8c95b-9d07-4c9e-adf4-7cd7d708eec9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2dfa1d5e59664af5a450bb1003ffbb4d', 'request_id': 'req-85e8c95b-9d07-4c9e-adf4-7cd7d708eec9'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:44.118070 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-85e8c95b-9d07-4c9e-adf4-7cd7d708eec9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2dfa1d5e59664af5a450bb1003ffbb4d', 'request_id': 'req-85e8c95b-9d07-4c9e-adf4-7cd7d708eec9'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:44.140706 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-85e8c95b-9d07-4c9e-adf4-7cd7d708eec9 admin admin] The token cache is being invalidate because project 2dfa1d5e59664af5a450bb1003ffbb4d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80382) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 09:18:44.141387 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-85e8c95b-9d07-4c9e-adf4-7cd7d708eec9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:44.143354 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-85e8c95b-9d07-4c9e-adf4-7cd7d708eec9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:44.149201 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 976/1935] 10.222.0.26 () {58 vars in 1232 bytes} [Thu Jul 14 09:18:43 2022] DELETE /identity/v3/projects/2dfa1d5e59664af5a450bb1003ffbb4d => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:18:44.155377 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abce007a-7c4c-4e2f-b979-8c7ce2f5d502 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:44.155738 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abce007a-7c4c-4e2f-b979-8c7ce2f5d502 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:44.156130 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abce007a-7c4c-4e2f-b979-8c7ce2f5d502 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:44.171858 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e8c95b-9d07-4c9e-adf4-7cd7d708eec9 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:44.175251 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-85e8c95b-9d07-4c9e-adf4-7cd7d708eec9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:18:44.180282 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-abce007a-7c4c-4e2f-b979-8c7ce2f5d502 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:18:44.181648 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-abce007a-7c4c-4e2f-b979-8c7ce2f5d502 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:18:44.296571 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 960/1936] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:18:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:18:44.304942 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3210d0-e3a2-4e33-8fc6-c5002c7ad0f0 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:44.305884 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3210d0-e3a2-4e33-8fc6-c5002c7ad0f0 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:44.307533 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3210d0-e3a2-4e33-8fc6-c5002c7ad0f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:44.334711 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e8c95b-9d07-4c9e-adf4-7cd7d708eec9 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:44.341876 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9addf2-9dd8-45dd-bda5-b706b139d521 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:44.343353 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9addf2-9dd8-45dd-bda5-b706b139d521 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:44.343657 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9addf2-9dd8-45dd-bda5-b706b139d521 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:44.343934 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9addf2-9dd8-45dd-bda5-b706b139d521 admin admin] PATH_INFO: `/v3/projects/44fd8600fda943249b00cccb59f24d0f` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:44.345791 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9addf2-9dd8-45dd-bda5-b706b139d521 admin admin] RBAC: Authorizing `identity:delete_project(project_id=44fd8600fda943249b00cccb59f24d0f)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:44.348316 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9addf2-9dd8-45dd-bda5-b706b139d521 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:44.363651 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-bc3210d0-e3a2-4e33-8fc6-c5002c7ad0f0 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:18:44.368982 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-bc3210d0-e3a2-4e33-8fc6-c5002c7ad0f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:18:44.391928 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 961/1937] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:18:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 09:18:44.398006 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3210d0-e3a2-4e33-8fc6-c5002c7ad0f0 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:44.418103 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3210d0-e3a2-4e33-8fc6-c5002c7ad0f0 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:44.437983 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-1f9addf2-9dd8-45dd-bda5-b706b139d521 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '44fd8600fda943249b00cccb59f24d0f', 'request_id': 'req-1f9addf2-9dd8-45dd-bda5-b706b139d521'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:44.447424 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-1f9addf2-9dd8-45dd-bda5-b706b139d521 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '44fd8600fda943249b00cccb59f24d0f', 'request_id': 'req-1f9addf2-9dd8-45dd-bda5-b706b139d521'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:44.460847 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-1f9addf2-9dd8-45dd-bda5-b706b139d521 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '44fd8600fda943249b00cccb59f24d0f', 'request_id': 'req-1f9addf2-9dd8-45dd-bda5-b706b139d521'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:44.487565 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-1f9addf2-9dd8-45dd-bda5-b706b139d521 admin admin] The token cache is being invalidate because project 44fd8600fda943249b00cccb59f24d0f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80382) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 09:18:44.488327 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-1f9addf2-9dd8-45dd-bda5-b706b139d521 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:44.492104 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-1f9addf2-9dd8-45dd-bda5-b706b139d521 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:44.494355 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 977/1938] 10.222.0.26 () {58 vars in 1232 bytes} [Thu Jul 14 09:18:44 2022] DELETE /identity/v3/projects/44fd8600fda943249b00cccb59f24d0f => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:18:44.499613 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba00ef03-c174-4624-ac86-dd2cbd584b91 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:44.500111 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba00ef03-c174-4624-ac86-dd2cbd584b91 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:44.500391 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba00ef03-c174-4624-ac86-dd2cbd584b91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:44.506352 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a464a8-9994-4a10-8137-ca3f21e2eb75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:44.507029 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a464a8-9994-4a10-8137-ca3f21e2eb75 admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:44.507365 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a464a8-9994-4a10-8137-ca3f21e2eb75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:44.507645 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a464a8-9994-4a10-8137-ca3f21e2eb75 admin admin] PATH_INFO: `/v3/domains` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:44.508434 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a464a8-9994-4a10-8137-ca3f21e2eb75 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:18:44.508738 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a464a8-9994-4a10-8137-ca3f21e2eb75 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:44.510978 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a464a8-9994-4a10-8137-ca3f21e2eb75 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:44.522494 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-ba00ef03-c174-4624-ac86-dd2cbd584b91 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:18:44.524412 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-ba00ef03-c174-4624-ac86-dd2cbd584b91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:18:44.525815 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 962/1939] 10.222.0.26 () {58 vars in 1152 bytes} [Thu Jul 14 09:18:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:18:44.536779 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a464a8-9994-4a10-8137-ca3f21e2eb75 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:44.538512 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-47a464a8-9994-4a10-8137-ca3f21e2eb75 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:18:44.561797 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 978/1940] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:18:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:18:44.563884 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09c0735-4507-480e-9f64-83616856c7d0 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:44.563884 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09c0735-4507-480e-9f64-83616856c7d0 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:44.564068 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09c0735-4507-480e-9f64-83616856c7d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:44.566154 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a464a8-9994-4a10-8137-ca3f21e2eb75 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:44.577254 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053e7795-e22a-4d04-8865-6dcc46e4dace admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:44.579338 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053e7795-e22a-4d04-8865-6dcc46e4dace admin admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:44.579840 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053e7795-e22a-4d04-8865-6dcc46e4dace admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:44.580326 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053e7795-e22a-4d04-8865-6dcc46e4dace admin admin] PATH_INFO: `/v3/projects` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:44.581989 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053e7795-e22a-4d04-8865-6dcc46e4dace admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:44.585215 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053e7795-e22a-4d04-8865-6dcc46e4dace admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:44.592054 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-e09c0735-4507-480e-9f64-83616856c7d0 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:18:44.593278 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-e09c0735-4507-480e-9f64-83616856c7d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:18:44.609568 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 979/1941] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:18:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:18:44.614770 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09c0735-4507-480e-9f64-83616856c7d0 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:44.621210 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:DELETE=9,keystone:SELECT=89,keystone:INSERT=2 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:18:44.637132 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09c0735-4507-480e-9f64-83616856c7d0 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:44.639040 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 963/1942] 10.222.0.26 () {60 vars in 1152 bytes} [Thu Jul 14 09:18:44 2022] POST /identity/v3/projects => generated 359 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 09:18:44.640214 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053e7795-e22a-4d04-8865-6dcc46e4dace admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:44.695031 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:18:44.713586 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21bb4d2-5045-406d-a74a-d1d77ebc142a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:44.714519 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21bb4d2-5045-406d-a74a-d1d77ebc142a admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:44.716095 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21bb4d2-5045-406d-a74a-d1d77ebc142a admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:44.716095 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21bb4d2-5045-406d-a74a-d1d77ebc142a admin admin] PATH_INFO: `/v3/domains` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:44.716095 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21bb4d2-5045-406d-a74a-d1d77ebc142a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:18:44.716476 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21bb4d2-5045-406d-a74a-d1d77ebc142a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:44.718166 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21bb4d2-5045-406d-a74a-d1d77ebc142a admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:44.731881 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053e7795-e22a-4d04-8865-6dcc46e4dace admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:44.737852 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 980/1943] 10.222.0.26 () {58 vars in 1152 bytes} [Thu Jul 14 09:18:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 09:18:44.738465 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2020e1a6-7087-4e35-b0e6-cfb6bb6ea2d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:44.739017 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21bb4d2-5045-406d-a74a-d1d77ebc142a admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:44.739163 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2020e1a6-7087-4e35-b0e6-cfb6bb6ea2d3 admin admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:44.740074 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2020e1a6-7087-4e35-b0e6-cfb6bb6ea2d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:44.740505 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2020e1a6-7087-4e35-b0e6-cfb6bb6ea2d3 admin admin] PATH_INFO: `/v3/users` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:44.741224 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2020e1a6-7087-4e35-b0e6-cfb6bb6ea2d3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:44.746293 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2020e1a6-7087-4e35-b0e6-cfb6bb6ea2d3 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:44.751245 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21bb4d2-5045-406d-a74a-d1d77ebc142a admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:44.758448 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff5cc2a-11f9-4a70-9ca8-6fa858e48654 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:44.759417 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff5cc2a-11f9-4a70-9ca8-6fa858e48654 admin admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:44.760060 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff5cc2a-11f9-4a70-9ca8-6fa858e48654 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:44.760437 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff5cc2a-11f9-4a70-9ca8-6fa858e48654 admin admin] PATH_INFO: `/v3/projects` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:44.761257 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff5cc2a-11f9-4a70-9ca8-6fa858e48654 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:44.763471 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff5cc2a-11f9-4a70-9ca8-6fa858e48654 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:44.769894 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 964/1944] 10.222.0.26 () {60 vars in 1143 bytes} [Thu Jul 14 09:18:44 2022] POST /identity/v3/users => generated 280 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 09:18:44.774526 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2020e1a6-7087-4e35-b0e6-cfb6bb6ea2d3 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:44.786901 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2020e1a6-7087-4e35-b0e6-cfb6bb6ea2d3 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:44.791984 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c570211-af20-4872-b508-d6a9c84bcd17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:44.792668 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c570211-af20-4872-b508-d6a9c84bcd17 admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:44.792957 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c570211-af20-4872-b508-d6a9c84bcd17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:44.793237 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c570211-af20-4872-b508-d6a9c84bcd17 admin admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:44.794022 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c570211-af20-4872-b508-d6a9c84bcd17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:44.795705 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c570211-af20-4872-b508-d6a9c84bcd17 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:44.796794 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 981/1945] 10.222.0.26 () {60 vars in 1152 bytes} [Thu Jul 14 09:18:44 2022] POST /identity/v3/projects => generated 371 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 09:18:44.800447 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff5cc2a-11f9-4a70-9ca8-6fa858e48654 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:44.809937 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 965/1946] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jul 14 09:18:44 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 09:18:44.811002 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c570211-af20-4872-b508-d6a9c84bcd17 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:44.821220 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c570211-af20-4872-b508-d6a9c84bcd17 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:44.876255 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff5cc2a-11f9-4a70-9ca8-6fa858e48654 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:44.888929 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96225925-87ef-4c1a-89b7-edced7992062 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:44.890327 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96225925-87ef-4c1a-89b7-edced7992062 admin admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:44.890614 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96225925-87ef-4c1a-89b7-edced7992062 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:44.890893 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96225925-87ef-4c1a-89b7-edced7992062 admin admin] PATH_INFO: `/v3/users` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:44.891732 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96225925-87ef-4c1a-89b7-edced7992062 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:44.894231 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96225925-87ef-4c1a-89b7-edced7992062 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:44.921808 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef51f77-0682-4eec-a61d-0830d030e7d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:44.923692 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef51f77-0682-4eec-a61d-0830d030e7d1 admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:44.923809 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef51f77-0682-4eec-a61d-0830d030e7d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:44.924098 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef51f77-0682-4eec-a61d-0830d030e7d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:44.927315 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef51f77-0682-4eec-a61d-0830d030e7d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:44.930550 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef51f77-0682-4eec-a61d-0830d030e7d1 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:44.939996 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 982/1947] 10.222.0.26 () {60 vars in 1143 bytes} [Thu Jul 14 09:18:44 2022] POST /identity/v3/users => generated 286 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 09:18:44.944814 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96225925-87ef-4c1a-89b7-edced7992062 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:44.957234 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96225925-87ef-4c1a-89b7-edced7992062 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:44.961838 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 966/1948] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jul 14 09:18:44 2022] GET /identity/v3/roles => generated 1420 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 09:18:44.966300 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef51f77-0682-4eec-a61d-0830d030e7d1 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:44.968601 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d895ae-dedd-440d-9cb3-a15de021767e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:44.969463 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d895ae-dedd-440d-9cb3-a15de021767e admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:44.969754 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d895ae-dedd-440d-9cb3-a15de021767e admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:44.970047 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d895ae-dedd-440d-9cb3-a15de021767e admin admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:44.970900 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d895ae-dedd-440d-9cb3-a15de021767e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:44.972696 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d895ae-dedd-440d-9cb3-a15de021767e admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:44.986350 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 983/1949] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jul 14 09:18:44 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 09:18:44.989655 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef51f77-0682-4eec-a61d-0830d030e7d1 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:44.993763 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d895ae-dedd-440d-9cb3-a15de021767e admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:45.008073 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be09e74-4e4e-44a9-baee-43f989659a2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:45.008988 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be09e74-4e4e-44a9-baee-43f989659a2e admin admin] REQUEST_METHOD: `PUT` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:45.009375 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be09e74-4e4e-44a9-baee-43f989659a2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:45.009538 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be09e74-4e4e-44a9-baee-43f989659a2e admin admin] PATH_INFO: `/v3/projects/b9f7890632b54d4ea23c4eb4567db821/users/8a8c5182c91743719e305f03f885479e/roles/d1691322ce6a4edab7ce722edf0f2988` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:45.019597 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d895ae-dedd-440d-9cb3-a15de021767e admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:45.033280 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe4efe6-c0b0-4a5f-8ba1-70d86b0660f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:45.033842 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe4efe6-c0b0-4a5f-8ba1-70d86b0660f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:45.034233 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe4efe6-c0b0-4a5f-8ba1-70d86b0660f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:45.034519 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe4efe6-c0b0-4a5f-8ba1-70d86b0660f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:45.035308 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe4efe6-c0b0-4a5f-8ba1-70d86b0660f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:45.036941 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe4efe6-c0b0-4a5f-8ba1-70d86b0660f6 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:45.043541 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be09e74-4e4e-44a9-baee-43f989659a2e admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9f7890632b54d4ea23c4eb4567db821, user_id=8a8c5182c91743719e305f03f885479e, role_id=d1691322ce6a4edab7ce722edf0f2988)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:45.046847 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be09e74-4e4e-44a9-baee-43f989659a2e admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:45.070216 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 984/1950] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jul 14 09:18:44 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 09:18:45.071480 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe4efe6-c0b0-4a5f-8ba1-70d86b0660f6 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:45.076384 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 967/1951] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jul 14 09:18:44 2022] PUT /identity/v3/projects/b9f7890632b54d4ea23c4eb4567db821/users/8a8c5182c91743719e305f03f885479e/roles/d1691322ce6a4edab7ce722edf0f2988 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:18:45.082746 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe4efe6-c0b0-4a5f-8ba1-70d86b0660f6 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:45.087027 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1071fb-329d-4898-8957-c1b7741991d9 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:45.087340 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1071fb-329d-4898-8957-c1b7741991d9 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:45.087701 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1071fb-329d-4898-8957-c1b7741991d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:45.122028 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-3a1071fb-329d-4898-8957-c1b7741991d9 None None] MFA Rules not processed for user `8a8c5182c91743719e305f03f885479e`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:18:45.123052 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-3a1071fb-329d-4898-8957-c1b7741991d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:18:45.166100 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678a3f55-606d-4ecf-aa47-3d6fd839b8b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:45.167855 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678a3f55-606d-4ecf-aa47-3d6fd839b8b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:45.168449 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678a3f55-606d-4ecf-aa47-3d6fd839b8b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:45.169259 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678a3f55-606d-4ecf-aa47-3d6fd839b8b2 admin admin] PATH_INFO: `/v3/projects/a09284b0d51f4148ace0c3cf1b58d266/users/ed3ee4b41b6e437a9e23a53be94f1455/roles/d1691322ce6a4edab7ce722edf0f2988` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:45.178685 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:18:45.193652 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678a3f55-606d-4ecf-aa47-3d6fd839b8b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a09284b0d51f4148ace0c3cf1b58d266, user_id=ed3ee4b41b6e437a9e23a53be94f1455, role_id=d1691322ce6a4edab7ce722edf0f2988)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:45.196375 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678a3f55-606d-4ecf-aa47-3d6fd839b8b2 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:45.209508 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 968/1952] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:18:45 2022] POST /identity/v3/auth/tokens => generated 3226 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:18:45.229202 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 985/1953] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jul 14 09:18:45 2022] PUT /identity/v3/projects/a09284b0d51f4148ace0c3cf1b58d266/users/ed3ee4b41b6e437a9e23a53be94f1455/roles/d1691322ce6a4edab7ce722edf0f2988 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:18:45.241083 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d518d8-d5d9-4b1d-a4be-bb7b7d5faa88 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:45.241516 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d518d8-d5d9-4b1d-a4be-bb7b7d5faa88 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:45.241686 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d518d8-d5d9-4b1d-a4be-bb7b7d5faa88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:45.260117 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678a3f55-606d-4ecf-aa47-3d6fd839b8b2 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:45.262097 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-678a3f55-606d-4ecf-aa47-3d6fd839b8b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:18:45.278352 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:18:45.295847 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-55d518d8-d5d9-4b1d-a4be-bb7b7d5faa88 None None] MFA Rules not processed for user `ed3ee4b41b6e437a9e23a53be94f1455`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:18:45.301864 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-55d518d8-d5d9-4b1d-a4be-bb7b7d5faa88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:18:45.357349 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678a3f55-606d-4ecf-aa47-3d6fd839b8b2 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:45.363243 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894f68fd-8195-4aa5-94d8-40ee3ed7a188 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e76f1952a5b146808844b9f2b7395035', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:45.364176 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894f68fd-8195-4aa5-94d8-40ee3ed7a188 service cinder] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:45.364621 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894f68fd-8195-4aa5-94d8-40ee3ed7a188 service cinder] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:45.365062 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894f68fd-8195-4aa5-94d8-40ee3ed7a188 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:45.405725 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 969/1954] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:18:45 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:18:45.418965 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d518d8-d5d9-4b1d-a4be-bb7b7d5faa88 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:45.420945 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-55d518d8-d5d9-4b1d-a4be-bb7b7d5faa88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:18:45.444849 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894f68fd-8195-4aa5-94d8-40ee3ed7a188 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:45.446744 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894f68fd-8195-4aa5-94d8-40ee3ed7a188 service cinder] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:45.457589 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 986/1955] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:18:45 2022] GET /identity/v3/auth/tokens => generated 3226 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 09:18:45.474442 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894f68fd-8195-4aa5-94d8-40ee3ed7a188 service cinder] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:45.484993 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d518d8-d5d9-4b1d-a4be-bb7b7d5faa88 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:45.491688 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c4a501-2e1b-403e-8274-e87d4f7c84f3 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '872dbb91c9e54f13b1a31fc8242126df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:45.492364 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c4a501-2e1b-403e-8274-e87d4f7c84f3 service glance] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:45.492649 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c4a501-2e1b-403e-8274-e87d4f7c84f3 service glance] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:45.492930 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c4a501-2e1b-403e-8274-e87d4f7c84f3 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:45.515007 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c4a501-2e1b-403e-8274-e87d4f7c84f3 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:45.517642 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c4a501-2e1b-403e-8274-e87d4f7c84f3 service glance] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:45.534610 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 970/1956] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:18:45 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 09:18:45.568812 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894f68fd-8195-4aa5-94d8-40ee3ed7a188 service cinder] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:45.573777 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de5417b-d839-4f4a-9544-bbedd424802c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:45.574830 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de5417b-d839-4f4a-9544-bbedd424802c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:45.575230 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de5417b-d839-4f4a-9544-bbedd424802c admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:45.575582 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de5417b-d839-4f4a-9544-bbedd424802c admin admin] PATH_INFO: `/v3/users/8a8c5182c91743719e305f03f885479e` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:45.576880 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de5417b-d839-4f4a-9544-bbedd424802c admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a8c5182c91743719e305f03f885479e)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:45.578931 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de5417b-d839-4f4a-9544-bbedd424802c admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:45.634813 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-6de5417b-d839-4f4a-9544-bbedd424802c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a8c5182c91743719e305f03f885479e'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:45.635734 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-6de5417b-d839-4f4a-9544-bbedd424802c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a8c5182c91743719e305f03f885479e'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:45.652309 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-6de5417b-d839-4f4a-9544-bbedd424802c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a8c5182c91743719e305f03f885479e'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:45.657312 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:18:45.665119 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-6de5417b-d839-4f4a-9544-bbedd424802c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a8c5182c91743719e305f03f885479e'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:45.666054 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-6de5417b-d839-4f4a-9544-bbedd424802c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a8c5182c91743719e305f03f885479e'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:45.682072 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 987/1957] 10.222.0.26 () {58 vars in 1223 bytes} [Thu Jul 14 09:18:45 2022] DELETE /identity/v3/users/8a8c5182c91743719e305f03f885479e => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 09:18:45.686677 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65a787b-54d9-488d-bbe8-9a1b9f38a15a None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:45.687111 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65a787b-54d9-488d-bbe8-9a1b9f38a15a None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:45.687542 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65a787b-54d9-488d-bbe8-9a1b9f38a15a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:45.706282 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-f65a787b-54d9-488d-bbe8-9a1b9f38a15a None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:18:45.707607 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-f65a787b-54d9-488d-bbe8-9a1b9f38a15a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:18:45.784836 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 971/1958] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:18:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 09:18:45.795429 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de5417b-d839-4f4a-9544-bbedd424802c admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:45.796941 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-6de5417b-d839-4f4a-9544-bbedd424802c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:18:45.857255 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de5417b-d839-4f4a-9544-bbedd424802c admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:45.861893 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4aa1d4-3299-4a1a-b660-c0fc625a28cc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '952911c492444ffea4b211a24b564117', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:45.862605 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4aa1d4-3299-4a1a-b660-c0fc625a28cc service neutron] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:45.862898 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4aa1d4-3299-4a1a-b660-c0fc625a28cc service neutron] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:45.863173 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4aa1d4-3299-4a1a-b660-c0fc625a28cc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:45.874283 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4aa1d4-3299-4a1a-b660-c0fc625a28cc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:45.875989 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4aa1d4-3299-4a1a-b660-c0fc625a28cc service neutron] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:45.885927 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 988/1959] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:18:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:18:46.179647 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65a787b-54d9-488d-bbe8-9a1b9f38a15a None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:46.181084 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-f65a787b-54d9-488d-bbe8-9a1b9f38a15a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:18:46.196915 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65a787b-54d9-488d-bbe8-9a1b9f38a15a None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:46.202707 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a4fb4f-aefc-4fb8-be38-3f934644a598 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:46.203807 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a4fb4f-aefc-4fb8-be38-3f934644a598 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:46.203910 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a4fb4f-aefc-4fb8-be38-3f934644a598 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:46.204293 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a4fb4f-aefc-4fb8-be38-3f934644a598 admin admin] PATH_INFO: `/v3/projects/b9f7890632b54d4ea23c4eb4567db821` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:46.205587 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a4fb4f-aefc-4fb8-be38-3f934644a598 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b9f7890632b54d4ea23c4eb4567db821)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:46.207583 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a4fb4f-aefc-4fb8-be38-3f934644a598 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:46.218256 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:18:46.260685 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-f3a4fb4f-aefc-4fb8-be38-3f934644a598 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b9f7890632b54d4ea23c4eb4567db821', 'request_id': 'req-f3a4fb4f-aefc-4fb8-be38-3f934644a598'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:46.267239 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-f3a4fb4f-aefc-4fb8-be38-3f934644a598 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b9f7890632b54d4ea23c4eb4567db821', 'request_id': 'req-f3a4fb4f-aefc-4fb8-be38-3f934644a598'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:46.273827 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-f3a4fb4f-aefc-4fb8-be38-3f934644a598 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b9f7890632b54d4ea23c4eb4567db821', 'request_id': 'req-f3a4fb4f-aefc-4fb8-be38-3f934644a598'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:46.287137 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-f3a4fb4f-aefc-4fb8-be38-3f934644a598 admin admin] The token cache is being invalidate because project b9f7890632b54d4ea23c4eb4567db821 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80381) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 09:18:46.287459 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-f3a4fb4f-aefc-4fb8-be38-3f934644a598 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:46.288104 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-f3a4fb4f-aefc-4fb8-be38-3f934644a598 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:46.289661 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 972/1960] 10.222.0.26 () {58 vars in 1232 bytes} [Thu Jul 14 09:18:46 2022] DELETE /identity/v3/projects/b9f7890632b54d4ea23c4eb4567db821 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:18:46.294795 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef52939e-b421-484d-b264-b4f240e63bd4 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:46.295280 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef52939e-b421-484d-b264-b4f240e63bd4 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:46.295369 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef52939e-b421-484d-b264-b4f240e63bd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:46.315786 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-ef52939e-b421-484d-b264-b4f240e63bd4 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:18:46.317039 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-ef52939e-b421-484d-b264-b4f240e63bd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:18:46.387532 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 989/1961] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:18:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 09:18:46.397569 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfb1b51-93de-4cc3-a14f-f3b401c67850 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:46.397880 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfb1b51-93de-4cc3-a14f-f3b401c67850 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:46.398238 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfb1b51-93de-4cc3-a14f-f3b401c67850 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:46.419470 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-fcfb1b51-93de-4cc3-a14f-f3b401c67850 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:18:46.420721 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-fcfb1b51-93de-4cc3-a14f-f3b401c67850 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:18:46.429130 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 973/1962] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:18:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:18:46.433710 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef52939e-b421-484d-b264-b4f240e63bd4 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:46.443924 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef52939e-b421-484d-b264-b4f240e63bd4 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:46.451769 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e301e33-4062-4eea-aef2-02d971db4a67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:46.452445 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e301e33-4062-4eea-aef2-02d971db4a67 admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:46.452729 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e301e33-4062-4eea-aef2-02d971db4a67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:46.453001 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e301e33-4062-4eea-aef2-02d971db4a67 admin admin] PATH_INFO: `/v3/domains` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:46.453758 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e301e33-4062-4eea-aef2-02d971db4a67 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:18:46.454063 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e301e33-4062-4eea-aef2-02d971db4a67 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:46.455737 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e301e33-4062-4eea-aef2-02d971db4a67 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:46.469478 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 990/1963] 10.222.0.26 () {58 vars in 1152 bytes} [Thu Jul 14 09:18:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:18:46.473819 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfb1b51-93de-4cc3-a14f-f3b401c67850 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:46.483474 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfb1b51-93de-4cc3-a14f-f3b401c67850 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:46.488282 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4161a026-09a0-46cb-b97c-08799aa6c666 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:46.488934 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4161a026-09a0-46cb-b97c-08799aa6c666 admin admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:46.489217 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4161a026-09a0-46cb-b97c-08799aa6c666 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:46.489496 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4161a026-09a0-46cb-b97c-08799aa6c666 admin admin] PATH_INFO: `/v3/projects` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:46.490190 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4161a026-09a0-46cb-b97c-08799aa6c666 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:46.492209 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4161a026-09a0-46cb-b97c-08799aa6c666 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:46.525267 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 974/1964] 10.222.0.26 () {60 vars in 1152 bytes} [Thu Jul 14 09:18:46 2022] POST /identity/v3/projects => generated 373 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 09:18:46.533898 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e301e33-4062-4eea-aef2-02d971db4a67 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:46.539771 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e301e33-4062-4eea-aef2-02d971db4a67 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:46.597505 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b513492-df43-42c9-b9ef-9a95f0300271 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:46.598166 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b513492-df43-42c9-b9ef-9a95f0300271 admin admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:46.598450 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b513492-df43-42c9-b9ef-9a95f0300271 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:46.598729 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b513492-df43-42c9-b9ef-9a95f0300271 admin admin] PATH_INFO: `/v3/users` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:46.599458 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b513492-df43-42c9-b9ef-9a95f0300271 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:46.601394 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b513492-df43-42c9-b9ef-9a95f0300271 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:46.622983 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 991/1965] 10.222.0.26 () {60 vars in 1143 bytes} [Thu Jul 14 09:18:46 2022] POST /identity/v3/users => generated 287 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 09:18:46.627244 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4161a026-09a0-46cb-b97c-08799aa6c666 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:46.638012 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4161a026-09a0-46cb-b97c-08799aa6c666 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:46.644038 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2ccdb4-babd-4f94-9bbf-0f1aef3d8720 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:46.644740 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2ccdb4-babd-4f94-9bbf-0f1aef3d8720 admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:46.645033 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2ccdb4-babd-4f94-9bbf-0f1aef3d8720 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:46.645649 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2ccdb4-babd-4f94-9bbf-0f1aef3d8720 admin admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:46.646420 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2ccdb4-babd-4f94-9bbf-0f1aef3d8720 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:46.648129 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2ccdb4-babd-4f94-9bbf-0f1aef3d8720 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:46.659035 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 975/1966] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jul 14 09:18:46 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 09:18:46.662858 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b513492-df43-42c9-b9ef-9a95f0300271 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:46.669406 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:18:46.673132 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b513492-df43-42c9-b9ef-9a95f0300271 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:46.680436 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f187ab0e-fe55-4a9b-a034-483924593470 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:46.681280 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f187ab0e-fe55-4a9b-a034-483924593470 admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:46.681596 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f187ab0e-fe55-4a9b-a034-483924593470 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:46.682032 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f187ab0e-fe55-4a9b-a034-483924593470 admin admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:46.683012 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f187ab0e-fe55-4a9b-a034-483924593470 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:46.684751 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f187ab0e-fe55-4a9b-a034-483924593470 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:46.695066 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 992/1967] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jul 14 09:18:46 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 09:18:46.699173 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2ccdb4-babd-4f94-9bbf-0f1aef3d8720 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:46.709761 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2ccdb4-babd-4f94-9bbf-0f1aef3d8720 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:46.714460 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cf5fa2-3f6b-4aa1-bcb4-fcc69e0430ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:46.715421 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cf5fa2-3f6b-4aa1-bcb4-fcc69e0430ec admin admin] REQUEST_METHOD: `PUT` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:46.715738 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cf5fa2-3f6b-4aa1-bcb4-fcc69e0430ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:46.716052 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cf5fa2-3f6b-4aa1-bcb4-fcc69e0430ec admin admin] PATH_INFO: `/v3/projects/1925ec1a2a3c45878a46109bdd449f5d/users/e786618ecbec401abd0cee49b6b407f4/roles/d1691322ce6a4edab7ce722edf0f2988` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:46.731364 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cf5fa2-3f6b-4aa1-bcb4-fcc69e0430ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=1925ec1a2a3c45878a46109bdd449f5d, user_id=e786618ecbec401abd0cee49b6b407f4, role_id=d1691322ce6a4edab7ce722edf0f2988)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:46.734077 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cf5fa2-3f6b-4aa1-bcb4-fcc69e0430ec admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:46.748350 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 976/1968] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jul 14 09:18:46 2022] PUT /identity/v3/projects/1925ec1a2a3c45878a46109bdd449f5d/users/e786618ecbec401abd0cee49b6b407f4/roles/d1691322ce6a4edab7ce722edf0f2988 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 09:18:46.758718 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdeee6e-148f-4aff-aaab-a0cd1dc67c67 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:46.759009 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdeee6e-148f-4aff-aaab-a0cd1dc67c67 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:46.759319 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdeee6e-148f-4aff-aaab-a0cd1dc67c67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:46.788871 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-9cdeee6e-148f-4aff-aaab-a0cd1dc67c67 None None] MFA Rules not processed for user `e786618ecbec401abd0cee49b6b407f4`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:18:46.790115 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-9cdeee6e-148f-4aff-aaab-a0cd1dc67c67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:18:46.857083 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 993/1969] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:18:46 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:18:46.867857 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cf5fa2-3f6b-4aa1-bcb4-fcc69e0430ec admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:46.869740 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-a2cf5fa2-3f6b-4aa1-bcb4-fcc69e0430ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:18:46.928001 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cf5fa2-3f6b-4aa1-bcb4-fcc69e0430ec admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:46.931844 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8bc6fc-b9b5-4450-a6be-5084f6487b3c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e76f1952a5b146808844b9f2b7395035', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:46.932487 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8bc6fc-b9b5-4450-a6be-5084f6487b3c service cinder] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:46.932772 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8bc6fc-b9b5-4450-a6be-5084f6487b3c service cinder] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:46.933051 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8bc6fc-b9b5-4450-a6be-5084f6487b3c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:46.941898 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8bc6fc-b9b5-4450-a6be-5084f6487b3c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:46.943365 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8bc6fc-b9b5-4450-a6be-5084f6487b3c service cinder] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:46.952391 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 977/1970] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:18:46 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:18:52.713390 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdeee6e-148f-4aff-aaab-a0cd1dc67c67 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:52.714911 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-9cdeee6e-148f-4aff-aaab-a0cd1dc67c67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:18:52.795283 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdeee6e-148f-4aff-aaab-a0cd1dc67c67 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:52.795283 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fefd6e-5550-4411-849c-5d3ae0a9fbb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:52.795283 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fefd6e-5550-4411-849c-5d3ae0a9fbb0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:52.795283 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fefd6e-5550-4411-849c-5d3ae0a9fbb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:52.795664 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fefd6e-5550-4411-849c-5d3ae0a9fbb0 admin admin] PATH_INFO: `/v3/users/7305968fdeca45499a983a343ea2751c` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:52.796306 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fefd6e-5550-4411-849c-5d3ae0a9fbb0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7305968fdeca45499a983a343ea2751c)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:52.798233 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fefd6e-5550-4411-849c-5d3ae0a9fbb0 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:52.856737 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-37fefd6e-5550-4411-849c-5d3ae0a9fbb0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7305968fdeca45499a983a343ea2751c'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:52.857519 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-37fefd6e-5550-4411-849c-5d3ae0a9fbb0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7305968fdeca45499a983a343ea2751c'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:52.870446 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-37fefd6e-5550-4411-849c-5d3ae0a9fbb0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7305968fdeca45499a983a343ea2751c'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:52.878966 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-37fefd6e-5550-4411-849c-5d3ae0a9fbb0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7305968fdeca45499a983a343ea2751c'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:52.879584 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-37fefd6e-5550-4411-849c-5d3ae0a9fbb0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7305968fdeca45499a983a343ea2751c'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:52.894345 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 994/1971] 10.222.0.26 () {58 vars in 1223 bytes} [Thu Jul 14 09:18:52 2022] DELETE /identity/v3/users/7305968fdeca45499a983a343ea2751c => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:18:52.899977 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c337e0-8e00-4eee-8e58-bd3ae33ebc92 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:52.900575 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c337e0-8e00-4eee-8e58-bd3ae33ebc92 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:52.900575 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c337e0-8e00-4eee-8e58-bd3ae33ebc92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:52.920270 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-48c337e0-8e00-4eee-8e58-bd3ae33ebc92 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:18:52.921485 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-48c337e0-8e00-4eee-8e58-bd3ae33ebc92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:18:52.951955 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:18:52.985192 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 978/1972] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:18:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:18:52.995047 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fefd6e-5550-4411-849c-5d3ae0a9fbb0 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:52.996616 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-37fefd6e-5550-4411-849c-5d3ae0a9fbb0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:18:53.003146 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:18:53.064962 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fefd6e-5550-4411-849c-5d3ae0a9fbb0 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:53.068630 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4369f3-6052-439d-a84a-a8d62ba43ca5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '952911c492444ffea4b211a24b564117', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:53.069291 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4369f3-6052-439d-a84a-a8d62ba43ca5 service neutron] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:53.069577 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4369f3-6052-439d-a84a-a8d62ba43ca5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:53.069856 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4369f3-6052-439d-a84a-a8d62ba43ca5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:53.079094 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4369f3-6052-439d-a84a-a8d62ba43ca5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:53.080570 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4369f3-6052-439d-a84a-a8d62ba43ca5 service neutron] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:53.090274 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 995/1973] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:18:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:18:53.385658 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c337e0-8e00-4eee-8e58-bd3ae33ebc92 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:53.387390 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-48c337e0-8e00-4eee-8e58-bd3ae33ebc92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:18:53.401136 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c337e0-8e00-4eee-8e58-bd3ae33ebc92 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:53.418007 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361c44eb-591b-4ba1-9734-eed594b9f5bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:53.419544 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361c44eb-591b-4ba1-9734-eed594b9f5bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:53.420109 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361c44eb-591b-4ba1-9734-eed594b9f5bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:53.420745 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361c44eb-591b-4ba1-9734-eed594b9f5bf admin admin] PATH_INFO: `/v3/projects/bc6ba239599f4512a6454e3c30d2a307` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:53.429152 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361c44eb-591b-4ba1-9734-eed594b9f5bf admin admin] RBAC: Authorizing `identity:delete_project(project_id=bc6ba239599f4512a6454e3c30d2a307)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:53.434916 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361c44eb-591b-4ba1-9734-eed594b9f5bf admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:53.513296 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-361c44eb-591b-4ba1-9734-eed594b9f5bf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bc6ba239599f4512a6454e3c30d2a307', 'request_id': 'req-361c44eb-591b-4ba1-9734-eed594b9f5bf'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:53.520266 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-361c44eb-591b-4ba1-9734-eed594b9f5bf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bc6ba239599f4512a6454e3c30d2a307', 'request_id': 'req-361c44eb-591b-4ba1-9734-eed594b9f5bf'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:53.527646 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-361c44eb-591b-4ba1-9734-eed594b9f5bf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bc6ba239599f4512a6454e3c30d2a307', 'request_id': 'req-361c44eb-591b-4ba1-9734-eed594b9f5bf'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:53.552450 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-361c44eb-591b-4ba1-9734-eed594b9f5bf admin admin] The token cache is being invalidate because project bc6ba239599f4512a6454e3c30d2a307 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80381) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 09:18:53.552450 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-361c44eb-591b-4ba1-9734-eed594b9f5bf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:53.552450 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-361c44eb-591b-4ba1-9734-eed594b9f5bf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:53.552450 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 979/1974] 10.222.0.26 () {58 vars in 1232 bytes} [Thu Jul 14 09:18:53 2022] DELETE /identity/v3/projects/bc6ba239599f4512a6454e3c30d2a307 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:18:53.554303 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4369f3-6052-439d-a84a-a8d62ba43ca5 service neutron] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:53.565349 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4369f3-6052-439d-a84a-a8d62ba43ca5 service neutron] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:53.632699 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e362308d-ca68-49e2-bc0d-247bf3423dc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:53.633705 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e362308d-ca68-49e2-bc0d-247bf3423dc9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:53.634089 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e362308d-ca68-49e2-bc0d-247bf3423dc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:53.634386 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e362308d-ca68-49e2-bc0d-247bf3423dc9 admin admin] PATH_INFO: `/v3/users/3d4ea23330a54ed5b2a6fb0f44f30a20` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:53.638050 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e362308d-ca68-49e2-bc0d-247bf3423dc9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3d4ea23330a54ed5b2a6fb0f44f30a20)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:53.640178 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e362308d-ca68-49e2-bc0d-247bf3423dc9 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:53.705010 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-e362308d-ca68-49e2-bc0d-247bf3423dc9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3d4ea23330a54ed5b2a6fb0f44f30a20'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:53.705670 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-e362308d-ca68-49e2-bc0d-247bf3423dc9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3d4ea23330a54ed5b2a6fb0f44f30a20'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:53.721504 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-e362308d-ca68-49e2-bc0d-247bf3423dc9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3d4ea23330a54ed5b2a6fb0f44f30a20'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:53.731390 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-e362308d-ca68-49e2-bc0d-247bf3423dc9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3d4ea23330a54ed5b2a6fb0f44f30a20'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:53.731744 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-e362308d-ca68-49e2-bc0d-247bf3423dc9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3d4ea23330a54ed5b2a6fb0f44f30a20'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:53.763907 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 996/1975] 10.222.0.26 () {58 vars in 1223 bytes} [Thu Jul 14 09:18:53 2022] DELETE /identity/v3/users/3d4ea23330a54ed5b2a6fb0f44f30a20 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:18:54.379307 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361c44eb-591b-4ba1-9734-eed594b9f5bf admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:54.382412 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-361c44eb-591b-4ba1-9734-eed594b9f5bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:18:54.413979 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361c44eb-591b-4ba1-9734-eed594b9f5bf admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:54.503255 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2d678b-bb82-425f-bfa5-4b3b78f11e53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:54.504996 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2d678b-bb82-425f-bfa5-4b3b78f11e53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:54.505347 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2d678b-bb82-425f-bfa5-4b3b78f11e53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:54.505632 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2d678b-bb82-425f-bfa5-4b3b78f11e53 admin admin] PATH_INFO: `/v3/projects/5cad85ac7d09413b828354895dbc88dd` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:54.507544 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2d678b-bb82-425f-bfa5-4b3b78f11e53 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5cad85ac7d09413b828354895dbc88dd)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:54.510214 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2d678b-bb82-425f-bfa5-4b3b78f11e53 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:54.577513 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-8f2d678b-bb82-425f-bfa5-4b3b78f11e53 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5cad85ac7d09413b828354895dbc88dd', 'request_id': 'req-8f2d678b-bb82-425f-bfa5-4b3b78f11e53'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:54.589137 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-8f2d678b-bb82-425f-bfa5-4b3b78f11e53 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5cad85ac7d09413b828354895dbc88dd', 'request_id': 'req-8f2d678b-bb82-425f-bfa5-4b3b78f11e53'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:54.600980 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:18:54.602471 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-8f2d678b-bb82-425f-bfa5-4b3b78f11e53 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5cad85ac7d09413b828354895dbc88dd', 'request_id': 'req-8f2d678b-bb82-425f-bfa5-4b3b78f11e53'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:54.617695 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-8f2d678b-bb82-425f-bfa5-4b3b78f11e53 admin admin] The token cache is being invalidate because project 5cad85ac7d09413b828354895dbc88dd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80381) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 09:18:54.623617 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-8f2d678b-bb82-425f-bfa5-4b3b78f11e53 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:54.623617 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-8f2d678b-bb82-425f-bfa5-4b3b78f11e53 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:54.623617 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 980/1976] 10.222.0.26 () {58 vars in 1232 bytes} [Thu Jul 14 09:18:54 2022] DELETE /identity/v3/projects/5cad85ac7d09413b828354895dbc88dd => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:18:54.625822 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949b3296-92ec-44dd-a2e2-644d336ae40b None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:54.626075 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949b3296-92ec-44dd-a2e2-644d336ae40b None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:54.626351 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949b3296-92ec-44dd-a2e2-644d336ae40b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:54.645442 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-949b3296-92ec-44dd-a2e2-644d336ae40b None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:18:54.647224 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-949b3296-92ec-44dd-a2e2-644d336ae40b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:18:54.667000 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:18:54.716366 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 997/1977] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:18:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:18:54.724079 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a58b89-cbd5-4dd1-93d1-e4fe8b942885 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:54.730830 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a58b89-cbd5-4dd1-93d1-e4fe8b942885 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:54.731294 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a58b89-cbd5-4dd1-93d1-e4fe8b942885 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:54.754902 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-46a58b89-cbd5-4dd1-93d1-e4fe8b942885 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:18:54.756190 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-46a58b89-cbd5-4dd1-93d1-e4fe8b942885 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:18:54.765738 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 981/1978] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:18:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:18:54.769889 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949b3296-92ec-44dd-a2e2-644d336ae40b None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:54.785142 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949b3296-92ec-44dd-a2e2-644d336ae40b None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:54.795577 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76771b62-2733-41a3-8ad8-e07244122807 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:54.796544 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76771b62-2733-41a3-8ad8-e07244122807 admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:54.797012 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76771b62-2733-41a3-8ad8-e07244122807 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:54.797537 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76771b62-2733-41a3-8ad8-e07244122807 admin admin] PATH_INFO: `/v3/domains` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:54.798453 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76771b62-2733-41a3-8ad8-e07244122807 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:18:54.798914 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76771b62-2733-41a3-8ad8-e07244122807 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:54.800725 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76771b62-2733-41a3-8ad8-e07244122807 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:54.814621 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 998/1979] 10.222.0.26 () {58 vars in 1152 bytes} [Thu Jul 14 09:18:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:18:54.824679 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a58b89-cbd5-4dd1-93d1-e4fe8b942885 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:54.829539 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a58b89-cbd5-4dd1-93d1-e4fe8b942885 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:54.837496 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92ece60-a2d4-4a3c-a181-4669ff41fc35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:54.837496 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92ece60-a2d4-4a3c-a181-4669ff41fc35 admin admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:54.837496 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92ece60-a2d4-4a3c-a181-4669ff41fc35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:54.837496 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92ece60-a2d4-4a3c-a181-4669ff41fc35 admin admin] PATH_INFO: `/v3/projects` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:54.838082 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92ece60-a2d4-4a3c-a181-4669ff41fc35 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:54.838858 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92ece60-a2d4-4a3c-a181-4669ff41fc35 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:54.870115 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 982/1980] 10.222.0.26 () {60 vars in 1152 bytes} [Thu Jul 14 09:18:54 2022] POST /identity/v3/projects => generated 369 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 09:18:54.874479 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76771b62-2733-41a3-8ad8-e07244122807 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:54.885467 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76771b62-2733-41a3-8ad8-e07244122807 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:54.957412 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18243360-c33f-4f7d-ac82-dacde2dbb326 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:54.958111 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18243360-c33f-4f7d-ac82-dacde2dbb326 admin admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:54.958393 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18243360-c33f-4f7d-ac82-dacde2dbb326 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:54.958666 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18243360-c33f-4f7d-ac82-dacde2dbb326 admin admin] PATH_INFO: `/v3/users` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:54.959400 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18243360-c33f-4f7d-ac82-dacde2dbb326 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:54.961344 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18243360-c33f-4f7d-ac82-dacde2dbb326 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:54.983825 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 999/1981] 10.222.0.26 () {60 vars in 1143 bytes} [Thu Jul 14 09:18:54 2022] POST /identity/v3/users => generated 285 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 09:18:54.987301 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92ece60-a2d4-4a3c-a181-4669ff41fc35 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:55.000496 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92ece60-a2d4-4a3c-a181-4669ff41fc35 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:55.005396 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94a1457-e644-4523-ae9c-a52ef3481f62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:55.006183 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94a1457-e644-4523-ae9c-a52ef3481f62 admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:55.006562 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94a1457-e644-4523-ae9c-a52ef3481f62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:55.006939 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94a1457-e644-4523-ae9c-a52ef3481f62 admin admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:55.008142 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94a1457-e644-4523-ae9c-a52ef3481f62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:55.009728 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94a1457-e644-4523-ae9c-a52ef3481f62 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:55.027248 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 983/1982] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jul 14 09:18:54 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 09:18:55.030575 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18243360-c33f-4f7d-ac82-dacde2dbb326 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:55.041800 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18243360-c33f-4f7d-ac82-dacde2dbb326 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:55.046757 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f2f360-2ce8-4f01-ae53-37fd4d5ae5f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:55.047451 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f2f360-2ce8-4f01-ae53-37fd4d5ae5f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:55.047738 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f2f360-2ce8-4f01-ae53-37fd4d5ae5f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:55.048014 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f2f360-2ce8-4f01-ae53-37fd4d5ae5f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:55.048757 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f2f360-2ce8-4f01-ae53-37fd4d5ae5f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:55.050355 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f2f360-2ce8-4f01-ae53-37fd4d5ae5f5 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:55.060816 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1000/1983] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jul 14 09:18:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 09:18:55.065267 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94a1457-e644-4523-ae9c-a52ef3481f62 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:55.078213 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94a1457-e644-4523-ae9c-a52ef3481f62 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:55.083075 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30123a9c-a85c-44ec-91f0-90a5541df041 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:55.084078 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30123a9c-a85c-44ec-91f0-90a5541df041 admin admin] REQUEST_METHOD: `PUT` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:55.084552 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30123a9c-a85c-44ec-91f0-90a5541df041 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:55.084836 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30123a9c-a85c-44ec-91f0-90a5541df041 admin admin] PATH_INFO: `/v3/projects/820d94ab92b841c38592454277033edb/users/b12fffb8eb3e4470b30ba478e6987443/roles/d1691322ce6a4edab7ce722edf0f2988` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:55.100635 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30123a9c-a85c-44ec-91f0-90a5541df041 admin admin] RBAC: Authorizing `identity:create_grant(project_id=820d94ab92b841c38592454277033edb, user_id=b12fffb8eb3e4470b30ba478e6987443, role_id=d1691322ce6a4edab7ce722edf0f2988)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:55.103109 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30123a9c-a85c-44ec-91f0-90a5541df041 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:55.119821 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 984/1984] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jul 14 09:18:55 2022] PUT /identity/v3/projects/820d94ab92b841c38592454277033edb/users/b12fffb8eb3e4470b30ba478e6987443/roles/d1691322ce6a4edab7ce722edf0f2988 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 09:18:55.135329 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db77dc78-71bb-4da6-9497-1bd807866da4 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:55.135753 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db77dc78-71bb-4da6-9497-1bd807866da4 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:55.136254 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db77dc78-71bb-4da6-9497-1bd807866da4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:55.167407 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-db77dc78-71bb-4da6-9497-1bd807866da4 None None] MFA Rules not processed for user `b12fffb8eb3e4470b30ba478e6987443`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:18:55.169969 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-db77dc78-71bb-4da6-9497-1bd807866da4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:18:55.261779 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1001/1985] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:18:55 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:18:55.263541 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30123a9c-a85c-44ec-91f0-90a5541df041 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:55.333580 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30123a9c-a85c-44ec-91f0-90a5541df041 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:55.339370 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3633116-f6f1-4e6f-b709-2f24e68cebe0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:55.340082 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3633116-f6f1-4e6f-b709-2f24e68cebe0 admin admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:55.340372 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3633116-f6f1-4e6f-b709-2f24e68cebe0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:55.340645 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3633116-f6f1-4e6f-b709-2f24e68cebe0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:55.341422 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3633116-f6f1-4e6f-b709-2f24e68cebe0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:55.343416 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3633116-f6f1-4e6f-b709-2f24e68cebe0 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:55.370773 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 985/1986] 10.222.0.26 () {60 vars in 1152 bytes} [Thu Jul 14 09:18:55 2022] POST /identity/v3/projects => generated 369 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 09:18:55.374949 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db77dc78-71bb-4da6-9497-1bd807866da4 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:55.385969 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db77dc78-71bb-4da6-9497-1bd807866da4 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:55.406484 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:18:55.455610 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ef3801-b090-410e-a3c8-7bbc16adf951 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:55.456338 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ef3801-b090-410e-a3c8-7bbc16adf951 admin admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:55.456631 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ef3801-b090-410e-a3c8-7bbc16adf951 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:55.456906 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ef3801-b090-410e-a3c8-7bbc16adf951 admin admin] PATH_INFO: `/v3/users` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:55.457651 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ef3801-b090-410e-a3c8-7bbc16adf951 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:55.459904 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ef3801-b090-410e-a3c8-7bbc16adf951 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:55.482176 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1002/1987] 10.222.0.26 () {60 vars in 1143 bytes} [Thu Jul 14 09:18:55 2022] POST /identity/v3/users => generated 291 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 09:18:55.486821 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3633116-f6f1-4e6f-b709-2f24e68cebe0 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:55.499663 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3633116-f6f1-4e6f-b709-2f24e68cebe0 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:55.507227 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5155c2d8-5c39-49fd-a23a-683333ed75e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:55.507227 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5155c2d8-5c39-49fd-a23a-683333ed75e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:55.507227 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5155c2d8-5c39-49fd-a23a-683333ed75e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:55.507227 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5155c2d8-5c39-49fd-a23a-683333ed75e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:55.507615 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5155c2d8-5c39-49fd-a23a-683333ed75e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:55.509772 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5155c2d8-5c39-49fd-a23a-683333ed75e5 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:55.522005 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 986/1988] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jul 14 09:18:55 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 09:18:55.526450 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ef3801-b090-410e-a3c8-7bbc16adf951 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:55.537173 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ef3801-b090-410e-a3c8-7bbc16adf951 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:55.542136 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1138f0-201d-48df-948c-d9ed9c06b3d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:55.543098 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1138f0-201d-48df-948c-d9ed9c06b3d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:55.543748 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1138f0-201d-48df-948c-d9ed9c06b3d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:55.544101 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1138f0-201d-48df-948c-d9ed9c06b3d6 admin admin] PATH_INFO: `/v3/projects/abf6d14f3134460bbd2235429f71f9a0/users/8858af43135e4314a4904824f8418f4a/roles/7b549d58d1724395b343609d895c80d0` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:55.566120 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1138f0-201d-48df-948c-d9ed9c06b3d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=abf6d14f3134460bbd2235429f71f9a0, user_id=8858af43135e4314a4904824f8418f4a, role_id=7b549d58d1724395b343609d895c80d0)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:55.568827 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1138f0-201d-48df-948c-d9ed9c06b3d6 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:55.584791 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1003/1989] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jul 14 09:18:55 2022] PUT /identity/v3/projects/abf6d14f3134460bbd2235429f71f9a0/users/8858af43135e4314a4904824f8418f4a/roles/7b549d58d1724395b343609d895c80d0 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 09:18:55.588795 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5155c2d8-5c39-49fd-a23a-683333ed75e5 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:55.599554 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5155c2d8-5c39-49fd-a23a-683333ed75e5 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:55.657223 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e594bf2-71b6-475e-90a2-1bf696203798 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:55.657891 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e594bf2-71b6-475e-90a2-1bf696203798 admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:55.658185 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e594bf2-71b6-475e-90a2-1bf696203798 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:55.658475 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e594bf2-71b6-475e-90a2-1bf696203798 admin admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:55.659224 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e594bf2-71b6-475e-90a2-1bf696203798 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:55.660821 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e594bf2-71b6-475e-90a2-1bf696203798 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:55.662136 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:18:55.672710 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 987/1990] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jul 14 09:18:55 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 09:18:55.676876 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1138f0-201d-48df-948c-d9ed9c06b3d6 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:55.692025 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1138f0-201d-48df-948c-d9ed9c06b3d6 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:55.701198 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60137206-d856-4522-b8b2-92905dfeeac9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:55.702383 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60137206-d856-4522-b8b2-92905dfeeac9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:55.702672 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60137206-d856-4522-b8b2-92905dfeeac9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:55.702951 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60137206-d856-4522-b8b2-92905dfeeac9 admin admin] PATH_INFO: `/v3/projects/abf6d14f3134460bbd2235429f71f9a0/users/8858af43135e4314a4904824f8418f4a/roles/7b549d58d1724395b343609d895c80d0` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:55.706831 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60137206-d856-4522-b8b2-92905dfeeac9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=abf6d14f3134460bbd2235429f71f9a0, user_id=8858af43135e4314a4904824f8418f4a, role_id=7b549d58d1724395b343609d895c80d0)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:55.709272 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60137206-d856-4522-b8b2-92905dfeeac9 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:55.719695 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1004/1991] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jul 14 09:18:55 2022] PUT /identity/v3/projects/abf6d14f3134460bbd2235429f71f9a0/users/8858af43135e4314a4904824f8418f4a/roles/7b549d58d1724395b343609d895c80d0 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:18:55.732238 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8447e32-0398-4f05-a9d0-335c7e195885 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:55.732493 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8447e32-0398-4f05-a9d0-335c7e195885 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:55.732765 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8447e32-0398-4f05-a9d0-335c7e195885 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:55.770480 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-e8447e32-0398-4f05-a9d0-335c7e195885 None None] MFA Rules not processed for user `8858af43135e4314a4904824f8418f4a`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:18:55.772056 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-e8447e32-0398-4f05-a9d0-335c7e195885 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:18:55.864298 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 988/1992] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:18:55 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:18:55.876643 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60137206-d856-4522-b8b2-92905dfeeac9 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:55.879944 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-60137206-d856-4522-b8b2-92905dfeeac9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:18:55.953097 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60137206-d856-4522-b8b2-92905dfeeac9 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:55.956920 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b850b327-e40d-4122-8db0-93009540330f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e76f1952a5b146808844b9f2b7395035', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:55.957756 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b850b327-e40d-4122-8db0-93009540330f service cinder] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:55.958108 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b850b327-e40d-4122-8db0-93009540330f service cinder] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:55.958569 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b850b327-e40d-4122-8db0-93009540330f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:55.968176 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b850b327-e40d-4122-8db0-93009540330f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:55.969719 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b850b327-e40d-4122-8db0-93009540330f service cinder] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:55.979678 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1005/1993] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:18:55 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:18:57.411792 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8447e32-0398-4f05-a9d0-335c7e195885 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:57.502517 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8447e32-0398-4f05-a9d0-335c7e195885 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:57.508725 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e79216-02be-4683-abdd-ecf228b3f4d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:57.510635 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e79216-02be-4683-abdd-ecf228b3f4d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:57.511308 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e79216-02be-4683-abdd-ecf228b3f4d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:57.511498 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e79216-02be-4683-abdd-ecf228b3f4d5 admin admin] PATH_INFO: `/v3/users/b12fffb8eb3e4470b30ba478e6987443` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:57.513378 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e79216-02be-4683-abdd-ecf228b3f4d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b12fffb8eb3e4470b30ba478e6987443)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:57.515960 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e79216-02be-4683-abdd-ecf228b3f4d5 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:57.603857 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-98e79216-02be-4683-abdd-ecf228b3f4d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b12fffb8eb3e4470b30ba478e6987443'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:57.612946 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-98e79216-02be-4683-abdd-ecf228b3f4d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b12fffb8eb3e4470b30ba478e6987443'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:57.613659 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-98e79216-02be-4683-abdd-ecf228b3f4d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b12fffb8eb3e4470b30ba478e6987443'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:57.614456 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-98e79216-02be-4683-abdd-ecf228b3f4d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b12fffb8eb3e4470b30ba478e6987443'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:57.622425 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-98e79216-02be-4683-abdd-ecf228b3f4d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b12fffb8eb3e4470b30ba478e6987443'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:57.640333 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:18:57.646899 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 989/1994] 10.222.0.26 () {58 vars in 1223 bytes} [Thu Jul 14 09:18:57 2022] DELETE /identity/v3/users/b12fffb8eb3e4470b30ba478e6987443 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:18:57.651605 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450e1e5f-e64d-435b-b21c-c4a31ba375fb None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:57.652116 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450e1e5f-e64d-435b-b21c-c4a31ba375fb None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:57.652116 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450e1e5f-e64d-435b-b21c-c4a31ba375fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:57.669862 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-450e1e5f-e64d-435b-b21c-c4a31ba375fb None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:18:57.671232 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-450e1e5f-e64d-435b-b21c-c4a31ba375fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:18:57.737008 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1006/1995] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:18:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 09:18:57.746638 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e79216-02be-4683-abdd-ecf228b3f4d5 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:57.748194 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-98e79216-02be-4683-abdd-ecf228b3f4d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:18:57.809097 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e79216-02be-4683-abdd-ecf228b3f4d5 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:57.812734 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01757e0-ee5e-4b87-b755-bc803fc6c675 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '952911c492444ffea4b211a24b564117', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:57.813390 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01757e0-ee5e-4b87-b755-bc803fc6c675 service neutron] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:57.813668 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01757e0-ee5e-4b87-b755-bc803fc6c675 service neutron] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:57.813943 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01757e0-ee5e-4b87-b755-bc803fc6c675 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:57.823270 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01757e0-ee5e-4b87-b755-bc803fc6c675 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:57.824723 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01757e0-ee5e-4b87-b755-bc803fc6c675 service neutron] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:57.835536 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 990/1996] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:18:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:18:58.120944 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450e1e5f-e64d-435b-b21c-c4a31ba375fb None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:58.122826 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-450e1e5f-e64d-435b-b21c-c4a31ba375fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:18:58.138059 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450e1e5f-e64d-435b-b21c-c4a31ba375fb None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:58.144147 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf68a352-ebeb-4095-b520-e58541381334 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:58.145439 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf68a352-ebeb-4095-b520-e58541381334 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:58.145720 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf68a352-ebeb-4095-b520-e58541381334 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:58.146317 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf68a352-ebeb-4095-b520-e58541381334 admin admin] PATH_INFO: `/v3/projects/820d94ab92b841c38592454277033edb` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:58.148034 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf68a352-ebeb-4095-b520-e58541381334 admin admin] RBAC: Authorizing `identity:delete_project(project_id=820d94ab92b841c38592454277033edb)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:58.150163 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf68a352-ebeb-4095-b520-e58541381334 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:58.168126 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:18:58.208126 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-cf68a352-ebeb-4095-b520-e58541381334 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '820d94ab92b841c38592454277033edb', 'request_id': 'req-cf68a352-ebeb-4095-b520-e58541381334'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:58.215205 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-cf68a352-ebeb-4095-b520-e58541381334 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '820d94ab92b841c38592454277033edb', 'request_id': 'req-cf68a352-ebeb-4095-b520-e58541381334'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:58.221115 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-cf68a352-ebeb-4095-b520-e58541381334 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '820d94ab92b841c38592454277033edb', 'request_id': 'req-cf68a352-ebeb-4095-b520-e58541381334'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:58.240298 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-cf68a352-ebeb-4095-b520-e58541381334 admin admin] The token cache is being invalidate because project 820d94ab92b841c38592454277033edb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80382) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 09:18:58.240573 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-cf68a352-ebeb-4095-b520-e58541381334 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:58.241364 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-cf68a352-ebeb-4095-b520-e58541381334 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:58.242977 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1007/1997] 10.222.0.26 () {58 vars in 1232 bytes} [Thu Jul 14 09:18:58 2022] DELETE /identity/v3/projects/820d94ab92b841c38592454277033edb => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:18:58.247075 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01757e0-ee5e-4b87-b755-bc803fc6c675 service neutron] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:58.257126 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01757e0-ee5e-4b87-b755-bc803fc6c675 service neutron] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:58.318860 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa5ee8c-62f9-4e18-a714-705a45c55afe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:58.319964 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa5ee8c-62f9-4e18-a714-705a45c55afe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:58.320251 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa5ee8c-62f9-4e18-a714-705a45c55afe admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:58.320529 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa5ee8c-62f9-4e18-a714-705a45c55afe admin admin] PATH_INFO: `/v3/users/8858af43135e4314a4904824f8418f4a` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:58.321824 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa5ee8c-62f9-4e18-a714-705a45c55afe admin admin] RBAC: Authorizing `identity:delete_user(user_id=8858af43135e4314a4904824f8418f4a)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:58.323827 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa5ee8c-62f9-4e18-a714-705a45c55afe admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:58.376091 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-cfa5ee8c-62f9-4e18-a714-705a45c55afe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8858af43135e4314a4904824f8418f4a'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:58.385338 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-cfa5ee8c-62f9-4e18-a714-705a45c55afe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8858af43135e4314a4904824f8418f4a'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:58.386119 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-cfa5ee8c-62f9-4e18-a714-705a45c55afe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8858af43135e4314a4904824f8418f4a'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:58.386728 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-cfa5ee8c-62f9-4e18-a714-705a45c55afe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8858af43135e4314a4904824f8418f4a'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:58.394519 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-cfa5ee8c-62f9-4e18-a714-705a45c55afe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8858af43135e4314a4904824f8418f4a'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:58.414113 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 991/1998] 10.222.0.26 () {58 vars in 1223 bytes} [Thu Jul 14 09:18:58 2022] DELETE /identity/v3/users/8858af43135e4314a4904824f8418f4a => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:18:58.648808 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf68a352-ebeb-4095-b520-e58541381334 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:58.650589 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-cf68a352-ebeb-4095-b520-e58541381334 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:18:58.668141 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf68a352-ebeb-4095-b520-e58541381334 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:58.752690 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa5ee8c-62f9-4e18-a714-705a45c55afe admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:58.754941 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-cfa5ee8c-62f9-4e18-a714-705a45c55afe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:18:58.766011 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089a8198-532a-4874-9463-b4715d46d034 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:58.767541 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089a8198-532a-4874-9463-b4715d46d034 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:58.768508 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089a8198-532a-4874-9463-b4715d46d034 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:58.768996 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089a8198-532a-4874-9463-b4715d46d034 admin admin] PATH_INFO: `/v3/users/e786618ecbec401abd0cee49b6b407f4` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:58.771775 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089a8198-532a-4874-9463-b4715d46d034 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e786618ecbec401abd0cee49b6b407f4)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:58.774793 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089a8198-532a-4874-9463-b4715d46d034 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:58.799859 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:18:58.848402 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-089a8198-532a-4874-9463-b4715d46d034 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e786618ecbec401abd0cee49b6b407f4'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:58.849113 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-089a8198-532a-4874-9463-b4715d46d034 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e786618ecbec401abd0cee49b6b407f4'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:58.852498 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa5ee8c-62f9-4e18-a714-705a45c55afe admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:58.857308 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce0d7ce-7a83-4947-b4b0-2a01385b7b46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:58.858185 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce0d7ce-7a83-4947-b4b0-2a01385b7b46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:58.858496 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce0d7ce-7a83-4947-b4b0-2a01385b7b46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:58.858771 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce0d7ce-7a83-4947-b4b0-2a01385b7b46 admin admin] PATH_INFO: `/v3/projects/abf6d14f3134460bbd2235429f71f9a0` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:58.860023 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce0d7ce-7a83-4947-b4b0-2a01385b7b46 admin admin] RBAC: Authorizing `identity:delete_project(project_id=abf6d14f3134460bbd2235429f71f9a0)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:58.862064 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce0d7ce-7a83-4947-b4b0-2a01385b7b46 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:58.872781 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-089a8198-532a-4874-9463-b4715d46d034 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e786618ecbec401abd0cee49b6b407f4'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:58.884900 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-089a8198-532a-4874-9463-b4715d46d034 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e786618ecbec401abd0cee49b6b407f4'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:58.886354 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-089a8198-532a-4874-9463-b4715d46d034 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e786618ecbec401abd0cee49b6b407f4'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:58.903285 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1008/1999] 10.222.0.26 () {58 vars in 1223 bytes} [Thu Jul 14 09:18:58 2022] DELETE /identity/v3/users/e786618ecbec401abd0cee49b6b407f4 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:18:58.907044 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c261ac-ee49-4040-8384-660fbdca9cde None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:58.907471 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c261ac-ee49-4040-8384-660fbdca9cde None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:58.908058 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c261ac-ee49-4040-8384-660fbdca9cde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:58.922116 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-bce0d7ce-7a83-4947-b4b0-2a01385b7b46 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'abf6d14f3134460bbd2235429f71f9a0', 'request_id': 'req-bce0d7ce-7a83-4947-b4b0-2a01385b7b46'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:58.932474 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-bce0d7ce-7a83-4947-b4b0-2a01385b7b46 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'abf6d14f3134460bbd2235429f71f9a0', 'request_id': 'req-bce0d7ce-7a83-4947-b4b0-2a01385b7b46'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:58.933127 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-77c261ac-ee49-4040-8384-660fbdca9cde None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:18:58.934451 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-77c261ac-ee49-4040-8384-660fbdca9cde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:18:58.947027 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-bce0d7ce-7a83-4947-b4b0-2a01385b7b46 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'abf6d14f3134460bbd2235429f71f9a0', 'request_id': 'req-bce0d7ce-7a83-4947-b4b0-2a01385b7b46'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:58.948694 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:18:58.961767 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-bce0d7ce-7a83-4947-b4b0-2a01385b7b46 admin admin] The token cache is being invalidate because project abf6d14f3134460bbd2235429f71f9a0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80381) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 09:18:58.962391 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-bce0d7ce-7a83-4947-b4b0-2a01385b7b46 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:58.963163 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-bce0d7ce-7a83-4947-b4b0-2a01385b7b46 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:58.965511 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 992/2000] 10.222.0.26 () {58 vars in 1232 bytes} [Thu Jul 14 09:18:58 2022] DELETE /identity/v3/projects/abf6d14f3134460bbd2235429f71f9a0 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:18:58.970813 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e69283-7b25-47d1-a361-a7d396e01689 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:58.971768 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e69283-7b25-47d1-a361-a7d396e01689 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:58.972241 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e69283-7b25-47d1-a361-a7d396e01689 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:58.997853 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-56e69283-7b25-47d1-a361-a7d396e01689 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:18:58.999180 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-56e69283-7b25-47d1-a361-a7d396e01689 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:18:59.027346 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1009/2001] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:18:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 09:18:59.034440 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c261ac-ee49-4040-8384-660fbdca9cde None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:59.036509 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-77c261ac-ee49-4040-8384-660fbdca9cde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:18:59.090377 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 993/2002] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:18:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 09:18:59.098219 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f2bce0-47a7-4f26-81b5-104ef3b3577d None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:59.098545 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f2bce0-47a7-4f26-81b5-104ef3b3577d None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:59.098909 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f2bce0-47a7-4f26-81b5-104ef3b3577d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:59.112676 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c261ac-ee49-4040-8384-660fbdca9cde None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:59.116577 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d67fe00-3d43-4772-a109-c289243dc680 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '952911c492444ffea4b211a24b564117', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:59.117315 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d67fe00-3d43-4772-a109-c289243dc680 service neutron] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:59.117655 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d67fe00-3d43-4772-a109-c289243dc680 service neutron] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:59.117925 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d67fe00-3d43-4772-a109-c289243dc680 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:59.124690 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-17f2bce0-47a7-4f26-81b5-104ef3b3577d None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:18:59.125952 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-17f2bce0-47a7-4f26-81b5-104ef3b3577d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:18:59.132402 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d67fe00-3d43-4772-a109-c289243dc680 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:59.134021 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d67fe00-3d43-4772-a109-c289243dc680 service neutron] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:59.139588 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 994/2003] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:18:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 09:18:59.148097 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f2bce0-47a7-4f26-81b5-104ef3b3577d None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:59.149671 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1010/2004] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:18:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 09:18:59.158742 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f2bce0-47a7-4f26-81b5-104ef3b3577d None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:59.165244 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a53273-bb12-4e25-83a2-60690389c89d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:59.166009 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a53273-bb12-4e25-83a2-60690389c89d admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:59.166364 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a53273-bb12-4e25-83a2-60690389c89d admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:59.166711 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a53273-bb12-4e25-83a2-60690389c89d admin admin] PATH_INFO: `/v3/domains` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:59.167604 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a53273-bb12-4e25-83a2-60690389c89d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:18:59.167983 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a53273-bb12-4e25-83a2-60690389c89d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:59.170898 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a53273-bb12-4e25-83a2-60690389c89d admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:59.182700 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 995/2005] 10.222.0.26 () {58 vars in 1152 bytes} [Thu Jul 14 09:18:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:18:59.187070 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d67fe00-3d43-4772-a109-c289243dc680 service neutron] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:59.197322 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d67fe00-3d43-4772-a109-c289243dc680 service neutron] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:59.203821 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5288f3a2-2d63-40bd-851d-33cc79be848a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:59.204797 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5288f3a2-2d63-40bd-851d-33cc79be848a admin admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:59.205479 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5288f3a2-2d63-40bd-851d-33cc79be848a admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:59.205479 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5288f3a2-2d63-40bd-851d-33cc79be848a admin admin] PATH_INFO: `/v3/projects` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:59.206108 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5288f3a2-2d63-40bd-851d-33cc79be848a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:59.208547 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5288f3a2-2d63-40bd-851d-33cc79be848a admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:59.241118 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1011/2006] 10.222.0.26 () {60 vars in 1152 bytes} [Thu Jul 14 09:18:59 2022] POST /identity/v3/projects => generated 381 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 09:18:59.245214 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a53273-bb12-4e25-83a2-60690389c89d admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:59.256591 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a53273-bb12-4e25-83a2-60690389c89d admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:59.308760 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:18:59.350879 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbba058-52d7-4a1f-830a-0023bedd0bdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:59.351718 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbba058-52d7-4a1f-830a-0023bedd0bdd admin admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:59.352025 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbba058-52d7-4a1f-830a-0023bedd0bdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:59.352303 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbba058-52d7-4a1f-830a-0023bedd0bdd admin admin] PATH_INFO: `/v3/users` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:59.353215 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbba058-52d7-4a1f-830a-0023bedd0bdd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:59.355335 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbba058-52d7-4a1f-830a-0023bedd0bdd admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:59.382386 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 996/2007] 10.222.0.26 () {60 vars in 1143 bytes} [Thu Jul 14 09:18:59 2022] POST /identity/v3/users => generated 291 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 09:18:59.387858 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5288f3a2-2d63-40bd-851d-33cc79be848a admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:59.404357 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5288f3a2-2d63-40bd-851d-33cc79be848a admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:59.410934 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af93078d-997e-4f85-b36d-6833ace7b74e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:59.411688 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af93078d-997e-4f85-b36d-6833ace7b74e admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:59.411978 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af93078d-997e-4f85-b36d-6833ace7b74e admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:59.412310 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af93078d-997e-4f85-b36d-6833ace7b74e admin admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:59.413045 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af93078d-997e-4f85-b36d-6833ace7b74e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:59.414713 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af93078d-997e-4f85-b36d-6833ace7b74e admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:59.432638 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1012/2008] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jul 14 09:18:59 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 09:18:59.433472 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbba058-52d7-4a1f-830a-0023bedd0bdd admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:59.447740 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbba058-52d7-4a1f-830a-0023bedd0bdd admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:59.454553 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9eccc1-57c0-4918-9943-18f5eeb922c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:59.455297 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9eccc1-57c0-4918-9943-18f5eeb922c6 admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:59.455590 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9eccc1-57c0-4918-9943-18f5eeb922c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:59.455870 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9eccc1-57c0-4918-9943-18f5eeb922c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:59.456904 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9eccc1-57c0-4918-9943-18f5eeb922c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:59.458542 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9eccc1-57c0-4918-9943-18f5eeb922c6 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:59.478074 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 997/2009] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jul 14 09:18:59 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 09:18:59.481983 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af93078d-997e-4f85-b36d-6833ace7b74e admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:59.505033 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af93078d-997e-4f85-b36d-6833ace7b74e admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:59.511820 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff40942-1ff3-4dc4-a232-e59077478f31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:59.514203 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff40942-1ff3-4dc4-a232-e59077478f31 admin admin] REQUEST_METHOD: `PUT` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:59.514613 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff40942-1ff3-4dc4-a232-e59077478f31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:59.514969 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff40942-1ff3-4dc4-a232-e59077478f31 admin admin] PATH_INFO: `/v3/projects/58be0432962d48429a67bf2656e43159/users/96d10cf2cbed4546b79637654b38f7d0/roles/d1691322ce6a4edab7ce722edf0f2988` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:59.533869 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff40942-1ff3-4dc4-a232-e59077478f31 admin admin] RBAC: Authorizing `identity:create_grant(project_id=58be0432962d48429a67bf2656e43159, user_id=96d10cf2cbed4546b79637654b38f7d0, role_id=d1691322ce6a4edab7ce722edf0f2988)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:59.536715 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff40942-1ff3-4dc4-a232-e59077478f31 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:59.563127 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1013/2010] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jul 14 09:18:59 2022] PUT /identity/v3/projects/58be0432962d48429a67bf2656e43159/users/96d10cf2cbed4546b79637654b38f7d0/roles/d1691322ce6a4edab7ce722edf0f2988 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:18:59.575049 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcab0714-1a75-423c-b782-2f02d5642ffa None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:59.575366 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcab0714-1a75-423c-b782-2f02d5642ffa None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:59.575704 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcab0714-1a75-423c-b782-2f02d5642ffa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:59.617120 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-dcab0714-1a75-423c-b782-2f02d5642ffa None None] MFA Rules not processed for user `96d10cf2cbed4546b79637654b38f7d0`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:18:59.620217 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-dcab0714-1a75-423c-b782-2f02d5642ffa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:18:59.683656 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff40942-1ff3-4dc4-a232-e59077478f31 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:59.685655 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-5ff40942-1ff3-4dc4-a232-e59077478f31 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:18:59.757670 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 998/2011] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:18:59 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:18:59.772253 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcab0714-1a75-423c-b782-2f02d5642ffa None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:59.839661 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff40942-1ff3-4dc4-a232-e59077478f31 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:59.850356 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7852a7c-18be-4774-a3f2-335710ba35ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:59.851368 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7852a7c-18be-4774-a3f2-335710ba35ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:59.851707 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7852a7c-18be-4774-a3f2-335710ba35ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:59.852119 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7852a7c-18be-4774-a3f2-335710ba35ad admin admin] PATH_INFO: `/v3/projects/1925ec1a2a3c45878a46109bdd449f5d` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:59.853343 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7852a7c-18be-4774-a3f2-335710ba35ad admin admin] RBAC: Authorizing `identity:delete_project(project_id=1925ec1a2a3c45878a46109bdd449f5d)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:59.855330 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7852a7c-18be-4774-a3f2-335710ba35ad admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:59.863087 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:18:59.870166 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcab0714-1a75-423c-b782-2f02d5642ffa None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:18:59.882116 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27128fc6-0eac-4a84-b55e-185ec3253106 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:18:59.882804 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27128fc6-0eac-4a84-b55e-185ec3253106 admin admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:18:59.883093 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27128fc6-0eac-4a84-b55e-185ec3253106 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:18:59.883418 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27128fc6-0eac-4a84-b55e-185ec3253106 admin admin] PATH_INFO: `/v3/projects` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:18:59.884083 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27128fc6-0eac-4a84-b55e-185ec3253106 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:18:59.885992 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27128fc6-0eac-4a84-b55e-185ec3253106 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:18:59.930930 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 999/2012] 10.222.0.26 () {60 vars in 1152 bytes} [Thu Jul 14 09:18:59 2022] POST /identity/v3/projects => generated 383 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 09:18:59.935274 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27128fc6-0eac-4a84-b55e-185ec3253106 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:18:59.960910 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-f7852a7c-18be-4774-a3f2-335710ba35ad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1925ec1a2a3c45878a46109bdd449f5d', 'request_id': 'req-f7852a7c-18be-4774-a3f2-335710ba35ad'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:59.969091 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-f7852a7c-18be-4774-a3f2-335710ba35ad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1925ec1a2a3c45878a46109bdd449f5d', 'request_id': 'req-f7852a7c-18be-4774-a3f2-335710ba35ad'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:18:59.975921 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:18:59.978619 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-f7852a7c-18be-4774-a3f2-335710ba35ad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1925ec1a2a3c45878a46109bdd449f5d', 'request_id': 'req-f7852a7c-18be-4774-a3f2-335710ba35ad'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:19:00.005169 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-f7852a7c-18be-4774-a3f2-335710ba35ad admin admin] The token cache is being invalidate because project 1925ec1a2a3c45878a46109bdd449f5d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80382) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 09:19:00.006080 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-f7852a7c-18be-4774-a3f2-335710ba35ad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:19:00.007441 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-f7852a7c-18be-4774-a3f2-335710ba35ad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:19:00.015093 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1014/2013] 10.222.0.26 () {58 vars in 1232 bytes} [Thu Jul 14 09:18:59 2022] DELETE /identity/v3/projects/1925ec1a2a3c45878a46109bdd449f5d => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 09:19:00.017683 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c5dc3c-672f-4f5c-af2d-da9335c41795 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:00.018351 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c5dc3c-672f-4f5c-af2d-da9335c41795 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:00.018845 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c5dc3c-672f-4f5c-af2d-da9335c41795 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:00.042190 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-70c5dc3c-672f-4f5c-af2d-da9335c41795 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:19:00.043606 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-70c5dc3c-672f-4f5c-af2d-da9335c41795 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:19:00.047971 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27128fc6-0eac-4a84-b55e-185ec3253106 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:19:00.190445 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1015/2014] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:19:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:19:00.199761 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865fb838-ebcd-4b49-bfa7-f6ef5df4d675 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:00.200385 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865fb838-ebcd-4b49-bfa7-f6ef5df4d675 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:00.200944 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865fb838-ebcd-4b49-bfa7-f6ef5df4d675 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:00.238679 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-865fb838-ebcd-4b49-bfa7-f6ef5df4d675 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:19:00.241969 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-865fb838-ebcd-4b49-bfa7-f6ef5df4d675 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:19:00.259141 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03944526-aa98-4a2f-9101-9400c94a1fac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:19:00.259910 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03944526-aa98-4a2f-9101-9400c94a1fac admin admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:00.260199 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03944526-aa98-4a2f-9101-9400c94a1fac admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:00.260485 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03944526-aa98-4a2f-9101-9400c94a1fac admin admin] PATH_INFO: `/v3/users` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:00.261646 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03944526-aa98-4a2f-9101-9400c94a1fac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:19:00.265792 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03944526-aa98-4a2f-9101-9400c94a1fac admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:19:00.269520 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1016/2015] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:19:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 09:19:00.276200 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865fb838-ebcd-4b49-bfa7-f6ef5df4d675 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:19:00.288388 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865fb838-ebcd-4b49-bfa7-f6ef5df4d675 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:19:00.298653 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c2890e-619e-4612-b1c2-0ad7136e7946 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:19:00.299408 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1000/2016] 10.222.0.26 () {60 vars in 1143 bytes} [Thu Jul 14 09:18:59 2022] POST /identity/v3/users => generated 298 bytes in 365 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 09:19:00.300207 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c2890e-619e-4612-b1c2-0ad7136e7946 admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:00.300490 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c2890e-619e-4612-b1c2-0ad7136e7946 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:00.300721 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c2890e-619e-4612-b1c2-0ad7136e7946 admin admin] PATH_INFO: `/v3/domains` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:00.301506 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c2890e-619e-4612-b1c2-0ad7136e7946 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:19:00.301814 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c2890e-619e-4612-b1c2-0ad7136e7946 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:19:00.305089 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03944526-aa98-4a2f-9101-9400c94a1fac admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:19:00.305267 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c2890e-619e-4612-b1c2-0ad7136e7946 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:19:00.308272 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-03944526-aa98-4a2f-9101-9400c94a1fac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:19:00.319242 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1017/2017] 10.222.0.26 () {58 vars in 1152 bytes} [Thu Jul 14 09:19:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:19:00.327127 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c2890e-619e-4612-b1c2-0ad7136e7946 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:19:00.330378 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03944526-aa98-4a2f-9101-9400c94a1fac admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:19:00.340380 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c2890e-619e-4612-b1c2-0ad7136e7946 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:19:00.348811 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6401b1cf-2a01-4c32-b105-bf1928076b12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:19:00.349655 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6401b1cf-2a01-4c32-b105-bf1928076b12 admin admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:00.349949 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6401b1cf-2a01-4c32-b105-bf1928076b12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:00.350227 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6401b1cf-2a01-4c32-b105-bf1928076b12 admin admin] PATH_INFO: `/v3/projects` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:00.350954 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6401b1cf-2a01-4c32-b105-bf1928076b12 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:19:00.351841 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9caba8-0033-4d23-9249-0db5859e3296 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:19:00.352782 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9caba8-0033-4d23-9249-0db5859e3296 admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:00.352963 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6401b1cf-2a01-4c32-b105-bf1928076b12 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:19:00.355867 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9caba8-0033-4d23-9249-0db5859e3296 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:00.356428 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9caba8-0033-4d23-9249-0db5859e3296 admin admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:00.357746 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9caba8-0033-4d23-9249-0db5859e3296 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:19:00.359826 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9caba8-0033-4d23-9249-0db5859e3296 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:19:00.372899 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1001/2018] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jul 14 09:19:00 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 09:19:00.377030 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9caba8-0033-4d23-9249-0db5859e3296 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:19:00.394206 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9caba8-0033-4d23-9249-0db5859e3296 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:19:00.398191 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1018/2019] 10.222.0.26 () {60 vars in 1152 bytes} [Thu Jul 14 09:19:00 2022] POST /identity/v3/projects => generated 375 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 09:19:00.405283 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6401b1cf-2a01-4c32-b105-bf1928076b12 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:19:00.475350 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e313a664-a3d3-4e6a-9177-ed016f2da1f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:19:00.476390 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e313a664-a3d3-4e6a-9177-ed016f2da1f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:00.476585 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e313a664-a3d3-4e6a-9177-ed016f2da1f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:00.476867 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e313a664-a3d3-4e6a-9177-ed016f2da1f3 admin admin] PATH_INFO: `/v3/projects/f15eb397cd5342af8b3c1d5a685d245a/users/bae569df7d73402c9c2717932b088786/roles/7b549d58d1724395b343609d895c80d0` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:00.478888 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:19:00.494676 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6401b1cf-2a01-4c32-b105-bf1928076b12 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:19:00.502194 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e313a664-a3d3-4e6a-9177-ed016f2da1f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f15eb397cd5342af8b3c1d5a685d245a, user_id=bae569df7d73402c9c2717932b088786, role_id=7b549d58d1724395b343609d895c80d0)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:19:00.502367 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82ce137-6480-41f4-bdc4-c3538b253d67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:19:00.502957 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82ce137-6480-41f4-bdc4-c3538b253d67 admin admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:00.503482 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82ce137-6480-41f4-bdc4-c3538b253d67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:00.503769 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82ce137-6480-41f4-bdc4-c3538b253d67 admin admin] PATH_INFO: `/v3/users` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:00.504565 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82ce137-6480-41f4-bdc4-c3538b253d67 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:19:00.505533 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e313a664-a3d3-4e6a-9177-ed016f2da1f3 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:19:00.506855 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82ce137-6480-41f4-bdc4-c3538b253d67 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:19:00.531977 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1002/2020] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jul 14 09:19:00 2022] PUT /identity/v3/projects/f15eb397cd5342af8b3c1d5a685d245a/users/bae569df7d73402c9c2717932b088786/roles/7b549d58d1724395b343609d895c80d0 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:19:00.535932 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e313a664-a3d3-4e6a-9177-ed016f2da1f3 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:19:00.540499 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1019/2021] 10.222.0.26 () {60 vars in 1143 bytes} [Thu Jul 14 09:19:00 2022] POST /identity/v3/users => generated 288 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 09:19:00.544657 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82ce137-6480-41f4-bdc4-c3538b253d67 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:19:00.558765 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82ce137-6480-41f4-bdc4-c3538b253d67 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:19:00.583282 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:19:00.618518 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e313a664-a3d3-4e6a-9177-ed016f2da1f3 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:19:00.624766 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f631bee-5822-4c78-8d5e-e54cd0e9ee4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:19:00.628135 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f631bee-5822-4c78-8d5e-e54cd0e9ee4b admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:00.628135 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f631bee-5822-4c78-8d5e-e54cd0e9ee4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:00.628785 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f631bee-5822-4c78-8d5e-e54cd0e9ee4b admin admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:00.629683 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f631bee-5822-4c78-8d5e-e54cd0e9ee4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:19:00.630695 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7552887d-fa8a-4786-bc02-107f2fb274c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:19:00.631375 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f631bee-5822-4c78-8d5e-e54cd0e9ee4b admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:19:00.631500 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7552887d-fa8a-4786-bc02-107f2fb274c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:00.631729 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7552887d-fa8a-4786-bc02-107f2fb274c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:00.632009 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7552887d-fa8a-4786-bc02-107f2fb274c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:00.632756 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7552887d-fa8a-4786-bc02-107f2fb274c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:19:00.634423 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7552887d-fa8a-4786-bc02-107f2fb274c7 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:19:00.644357 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1003/2022] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jul 14 09:19:00 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 09:19:00.648845 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f631bee-5822-4c78-8d5e-e54cd0e9ee4b admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:19:00.651545 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1020/2023] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jul 14 09:19:00 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 09:19:00.655570 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7552887d-fa8a-4786-bc02-107f2fb274c7 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:19:00.663425 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f631bee-5822-4c78-8d5e-e54cd0e9ee4b admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:19:00.665422 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7552887d-fa8a-4786-bc02-107f2fb274c7 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:19:00.670768 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df394409-2cf8-4fb8-bad8-a81974ebe0b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:19:00.670920 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b99126-0e90-4ff4-b6c3-dc3ec3384d3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:19:00.671526 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b99126-0e90-4ff4-b6c3-dc3ec3384d3b admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:00.671712 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df394409-2cf8-4fb8-bad8-a81974ebe0b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:00.671817 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b99126-0e90-4ff4-b6c3-dc3ec3384d3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:00.672006 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df394409-2cf8-4fb8-bad8-a81974ebe0b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:00.672114 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b99126-0e90-4ff4-b6c3-dc3ec3384d3b admin admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:00.672288 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df394409-2cf8-4fb8-bad8-a81974ebe0b7 admin admin] PATH_INFO: `/v3/projects/f15eb397cd5342af8b3c1d5a685d245a/users/bae569df7d73402c9c2717932b088786/roles/7b549d58d1724395b343609d895c80d0` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:00.672794 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b99126-0e90-4ff4-b6c3-dc3ec3384d3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:19:00.674338 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df394409-2cf8-4fb8-bad8-a81974ebe0b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f15eb397cd5342af8b3c1d5a685d245a, user_id=bae569df7d73402c9c2717932b088786, role_id=7b549d58d1724395b343609d895c80d0)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:19:00.674460 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b99126-0e90-4ff4-b6c3-dc3ec3384d3b admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:19:00.677091 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df394409-2cf8-4fb8-bad8-a81974ebe0b7 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:19:00.687430 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1021/2024] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jul 14 09:19:00 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 09:19:00.691730 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b99126-0e90-4ff4-b6c3-dc3ec3384d3b admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:19:00.705660 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1004/2025] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jul 14 09:19:00 2022] PUT /identity/v3/projects/f15eb397cd5342af8b3c1d5a685d245a/users/bae569df7d73402c9c2717932b088786/roles/7b549d58d1724395b343609d895c80d0 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:19:00.706207 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b99126-0e90-4ff4-b6c3-dc3ec3384d3b admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:19:00.707464 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84790c9-dcd8-40b8-bd1e-6acd58c3dde1 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:00.707843 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84790c9-dcd8-40b8-bd1e-6acd58c3dde1 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:00.708141 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84790c9-dcd8-40b8-bd1e-6acd58c3dde1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:00.750661 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-e84790c9-dcd8-40b8-bd1e-6acd58c3dde1 None None] MFA Rules not processed for user `bae569df7d73402c9c2717932b088786`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:19:00.752087 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-e84790c9-dcd8-40b8-bd1e-6acd58c3dde1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:19:00.784084 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7193b3fc-f934-4365-99cb-9ee491664ee9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:19:00.784994 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7193b3fc-f934-4365-99cb-9ee491664ee9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:00.785284 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7193b3fc-f934-4365-99cb-9ee491664ee9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:00.785561 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7193b3fc-f934-4365-99cb-9ee491664ee9 admin admin] PATH_INFO: `/v3/projects/7a5344011639483e98b89356e40d8d81/users/38970dab4776492a8f2fccaf9705ebc0/roles/d1691322ce6a4edab7ce722edf0f2988` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:00.803233 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7193b3fc-f934-4365-99cb-9ee491664ee9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a5344011639483e98b89356e40d8d81, user_id=38970dab4776492a8f2fccaf9705ebc0, role_id=d1691322ce6a4edab7ce722edf0f2988)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:19:00.806183 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7193b3fc-f934-4365-99cb-9ee491664ee9 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:19:00.829733 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1022/2026] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jul 14 09:19:00 2022] PUT /identity/v3/projects/7a5344011639483e98b89356e40d8d81/users/38970dab4776492a8f2fccaf9705ebc0/roles/d1691322ce6a4edab7ce722edf0f2988 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 09:19:00.839913 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7707963-6357-45ee-b581-8f792eb53678 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:00.840207 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7707963-6357-45ee-b581-8f792eb53678 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:00.840533 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7707963-6357-45ee-b581-8f792eb53678 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:00.863096 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1005/2027] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:19:00 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:19:00.871771 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84790c9-dcd8-40b8-bd1e-6acd58c3dde1 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:19:00.873470 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-e84790c9-dcd8-40b8-bd1e-6acd58c3dde1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:19:00.881766 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-d7707963-6357-45ee-b581-8f792eb53678 None None] MFA Rules not processed for user `38970dab4776492a8f2fccaf9705ebc0`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:19:00.883322 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-d7707963-6357-45ee-b581-8f792eb53678 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:19:00.971230 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84790c9-dcd8-40b8-bd1e-6acd58c3dde1 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:19:00.986277 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:19:01.042098 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1023/2028] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:19:00 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 09:19:01.055004 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957c76fb-2247-466b-9741-a94f5839c289 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e76f1952a5b146808844b9f2b7395035', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:19:01.055498 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7707963-6357-45ee-b581-8f792eb53678 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:19:01.055889 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957c76fb-2247-466b-9741-a94f5839c289 service cinder] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:01.056221 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957c76fb-2247-466b-9741-a94f5839c289 service cinder] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:01.056587 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957c76fb-2247-466b-9741-a94f5839c289 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:01.060227 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-d7707963-6357-45ee-b581-8f792eb53678 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:19:01.078900 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:19:01.081576 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7707963-6357-45ee-b581-8f792eb53678 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:19:01.090277 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e096ceb-e05b-406c-ad8e-0030d2b2115e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e76f1952a5b146808844b9f2b7395035', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:19:01.091490 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e096ceb-e05b-406c-ad8e-0030d2b2115e service cinder] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:01.091837 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e096ceb-e05b-406c-ad8e-0030d2b2115e service cinder] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:01.092175 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e096ceb-e05b-406c-ad8e-0030d2b2115e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:01.105240 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e096ceb-e05b-406c-ad8e-0030d2b2115e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:19:01.107101 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e096ceb-e05b-406c-ad8e-0030d2b2115e service cinder] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:19:01.139574 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1024/2029] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:19:01 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:19:01.165345 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957c76fb-2247-466b-9741-a94f5839c289 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:19:01.166964 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957c76fb-2247-466b-9741-a94f5839c289 service cinder] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:19:01.195027 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1006/2030] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:19:00 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 09:19:01.318824 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e096ceb-e05b-406c-ad8e-0030d2b2115e service cinder] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:19:01.332946 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e096ceb-e05b-406c-ad8e-0030d2b2115e service cinder] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:19:01.337459 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85e6add-a1f6-4564-a24d-3d0f5b22d55a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e76f1952a5b146808844b9f2b7395035', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:19:01.338117 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85e6add-a1f6-4564-a24d-3d0f5b22d55a service cinder] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:01.338615 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85e6add-a1f6-4564-a24d-3d0f5b22d55a service cinder] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:01.338895 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85e6add-a1f6-4564-a24d-3d0f5b22d55a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:01.340884 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-e85e6add-a1f6-4564-a24d-3d0f5b22d55a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:19:01.442409 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85e6add-a1f6-4564-a24d-3d0f5b22d55a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:19:01.452215 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85e6add-a1f6-4564-a24d-3d0f5b22d55a service cinder] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:19:01.483452 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1025/2031] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:19:01 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:19:01.650853 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957c76fb-2247-466b-9741-a94f5839c289 service cinder] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:19:01.656425 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-957c76fb-2247-466b-9741-a94f5839c289 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:19:01.753858 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957c76fb-2247-466b-9741-a94f5839c289 service cinder] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:19:01.758856 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88a01bb-cb7a-4dd2-8c0f-58290ce8d01f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:19:01.759772 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88a01bb-cb7a-4dd2-8c0f-58290ce8d01f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:01.760053 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88a01bb-cb7a-4dd2-8c0f-58290ce8d01f admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:01.760324 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88a01bb-cb7a-4dd2-8c0f-58290ce8d01f admin admin] PATH_INFO: `/v3/users/854fc6f464c940a78f779438aa3ee8a9` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:01.761768 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88a01bb-cb7a-4dd2-8c0f-58290ce8d01f admin admin] RBAC: Authorizing `identity:delete_user(user_id=854fc6f464c940a78f779438aa3ee8a9)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:19:01.763972 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88a01bb-cb7a-4dd2-8c0f-58290ce8d01f admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:19:01.820379 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-f88a01bb-cb7a-4dd2-8c0f-58290ce8d01f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '854fc6f464c940a78f779438aa3ee8a9'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:19:01.828808 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-f88a01bb-cb7a-4dd2-8c0f-58290ce8d01f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '854fc6f464c940a78f779438aa3ee8a9'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:19:01.829567 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-f88a01bb-cb7a-4dd2-8c0f-58290ce8d01f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '854fc6f464c940a78f779438aa3ee8a9'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:19:01.830213 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-f88a01bb-cb7a-4dd2-8c0f-58290ce8d01f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '854fc6f464c940a78f779438aa3ee8a9'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:19:01.837640 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-f88a01bb-cb7a-4dd2-8c0f-58290ce8d01f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '854fc6f464c940a78f779438aa3ee8a9'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:19:01.846002 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85e6add-a1f6-4564-a24d-3d0f5b22d55a service cinder] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:19:01.856340 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1007/2032] 10.222.0.26 () {58 vars in 1223 bytes} [Thu Jul 14 09:19:01 2022] DELETE /identity/v3/users/854fc6f464c940a78f779438aa3ee8a9 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:19:01.856753 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85e6add-a1f6-4564-a24d-3d0f5b22d55a service cinder] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:19:01.918768 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ece4036-8fcd-4905-b192-e99e3c181c8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:19:01.920766 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ece4036-8fcd-4905-b192-e99e3c181c8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:01.924264 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ece4036-8fcd-4905-b192-e99e3c181c8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:01.924786 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ece4036-8fcd-4905-b192-e99e3c181c8f admin admin] PATH_INFO: `/v3/users/96d10cf2cbed4546b79637654b38f7d0` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:01.926702 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ece4036-8fcd-4905-b192-e99e3c181c8f admin admin] RBAC: Authorizing `identity:delete_user(user_id=96d10cf2cbed4546b79637654b38f7d0)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:19:01.929178 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ece4036-8fcd-4905-b192-e99e3c181c8f admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:19:01.990001 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-7ece4036-8fcd-4905-b192-e99e3c181c8f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '96d10cf2cbed4546b79637654b38f7d0'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:19:01.990826 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-7ece4036-8fcd-4905-b192-e99e3c181c8f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '96d10cf2cbed4546b79637654b38f7d0'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:19:02.004496 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-7ece4036-8fcd-4905-b192-e99e3c181c8f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '96d10cf2cbed4546b79637654b38f7d0'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:19:02.007424 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:19:02.022395 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-7ece4036-8fcd-4905-b192-e99e3c181c8f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '96d10cf2cbed4546b79637654b38f7d0'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:19:02.024078 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-7ece4036-8fcd-4905-b192-e99e3c181c8f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '96d10cf2cbed4546b79637654b38f7d0'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:19:02.042125 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1026/2033] 10.222.0.26 () {58 vars in 1223 bytes} [Thu Jul 14 09:19:01 2022] DELETE /identity/v3/users/96d10cf2cbed4546b79637654b38f7d0 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 09:19:02.044349 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5017551-c876-45d9-b715-780b761e89cb None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:02.044639 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5017551-c876-45d9-b715-780b761e89cb None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:02.044921 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5017551-c876-45d9-b715-780b761e89cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:02.057521 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ece4036-8fcd-4905-b192-e99e3c181c8f admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:19:02.059000 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-7ece4036-8fcd-4905-b192-e99e3c181c8f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:19:02.069616 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-b5017551-c876-45d9-b715-780b761e89cb None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:19:02.070894 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-b5017551-c876-45d9-b715-780b761e89cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:19:02.099532 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:19:02.133691 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ece4036-8fcd-4905-b192-e99e3c181c8f admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:19:02.138406 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10906beb-c89b-43bb-891f-5149b3d1f70a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:19:02.139064 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1008/2034] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:19:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:19:02.139437 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10906beb-c89b-43bb-891f-5149b3d1f70a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:02.139803 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10906beb-c89b-43bb-891f-5149b3d1f70a admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:02.140152 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10906beb-c89b-43bb-891f-5149b3d1f70a admin admin] PATH_INFO: `/v3/projects/836b616f05a9421fbc776498f4ce7bd0` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:02.141609 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10906beb-c89b-43bb-891f-5149b3d1f70a admin admin] RBAC: Authorizing `identity:delete_project(project_id=836b616f05a9421fbc776498f4ce7bd0)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:19:02.143783 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10906beb-c89b-43bb-891f-5149b3d1f70a admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:19:02.149746 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5017551-c876-45d9-b715-780b761e89cb None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:19:02.151122 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-b5017551-c876-45d9-b715-780b761e89cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:19:02.193004 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-10906beb-c89b-43bb-891f-5149b3d1f70a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '836b616f05a9421fbc776498f4ce7bd0', 'request_id': 'req-10906beb-c89b-43bb-891f-5149b3d1f70a'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:19:02.199071 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-10906beb-c89b-43bb-891f-5149b3d1f70a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '836b616f05a9421fbc776498f4ce7bd0', 'request_id': 'req-10906beb-c89b-43bb-891f-5149b3d1f70a'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:19:02.204777 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-10906beb-c89b-43bb-891f-5149b3d1f70a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '836b616f05a9421fbc776498f4ce7bd0', 'request_id': 'req-10906beb-c89b-43bb-891f-5149b3d1f70a'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:19:02.210588 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5017551-c876-45d9-b715-780b761e89cb None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:19:02.218150 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-10906beb-c89b-43bb-891f-5149b3d1f70a admin admin] The token cache is being invalidate because project 836b616f05a9421fbc776498f4ce7bd0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80382) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 09:19:02.218434 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-10906beb-c89b-43bb-891f-5149b3d1f70a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:19:02.219059 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-10906beb-c89b-43bb-891f-5149b3d1f70a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:19:02.220672 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1027/2035] 10.222.0.26 () {58 vars in 1232 bytes} [Thu Jul 14 09:19:02 2022] DELETE /identity/v3/projects/836b616f05a9421fbc776498f4ce7bd0 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 09:19:02.225457 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330f8603-3ef4-4375-bf21-e56f52d8783a None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:02.225829 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330f8603-3ef4-4375-bf21-e56f52d8783a None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:02.226105 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330f8603-3ef4-4375-bf21-e56f52d8783a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:02.244928 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-330f8603-3ef4-4375-bf21-e56f52d8783a None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:19:02.246320 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-330f8603-3ef4-4375-bf21-e56f52d8783a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:19:02.265590 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f0b9a7-6648-4d81-9891-123e601b002a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '952911c492444ffea4b211a24b564117', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:19:02.266285 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f0b9a7-6648-4d81-9891-123e601b002a service neutron] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:02.266577 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f0b9a7-6648-4d81-9891-123e601b002a service neutron] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:02.266860 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f0b9a7-6648-4d81-9891-123e601b002a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:02.268836 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-f6f0b9a7-6648-4d81-9891-123e601b002a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:19:02.309904 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1028/2036] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:19:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 09:19:02.319019 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa020a7-1c87-4f7e-bb0d-a348f2f2dbba None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:02.319348 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa020a7-1c87-4f7e-bb0d-a348f2f2dbba None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:02.319620 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa020a7-1c87-4f7e-bb0d-a348f2f2dbba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:02.339384 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-7aa020a7-1c87-4f7e-bb0d-a348f2f2dbba None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:19:02.340588 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-7aa020a7-1c87-4f7e-bb0d-a348f2f2dbba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:19:02.342064 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f0b9a7-6648-4d81-9891-123e601b002a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:19:02.343693 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f0b9a7-6648-4d81-9891-123e601b002a service neutron] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:19:02.348567 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1029/2037] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:19:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 09:19:02.352746 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa020a7-1c87-4f7e-bb0d-a348f2f2dbba None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:19:02.354219 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1009/2038] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:19:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 09:19:02.362387 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa020a7-1c87-4f7e-bb0d-a348f2f2dbba None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:19:02.367039 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ecdf71-f9c0-4a2b-a7c0-0c67a1877e92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:19:02.367735 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ecdf71-f9c0-4a2b-a7c0-0c67a1877e92 admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:02.368016 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ecdf71-f9c0-4a2b-a7c0-0c67a1877e92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:02.368286 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ecdf71-f9c0-4a2b-a7c0-0c67a1877e92 admin admin] PATH_INFO: `/v3/domains` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:02.369044 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ecdf71-f9c0-4a2b-a7c0-0c67a1877e92 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:19:02.369348 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ecdf71-f9c0-4a2b-a7c0-0c67a1877e92 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:19:02.370966 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ecdf71-f9c0-4a2b-a7c0-0c67a1877e92 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:19:02.374008 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:19:02.383998 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1030/2039] 10.222.0.26 () {58 vars in 1152 bytes} [Thu Jul 14 09:19:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 09:19:02.389299 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f0b9a7-6648-4d81-9891-123e601b002a service neutron] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:19:02.402585 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f0b9a7-6648-4d81-9891-123e601b002a service neutron] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:19:02.409315 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4fe722-b44a-457e-806a-de065aba531b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:19:02.410318 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4fe722-b44a-457e-806a-de065aba531b admin admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:02.410860 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4fe722-b44a-457e-806a-de065aba531b admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:02.411294 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4fe722-b44a-457e-806a-de065aba531b admin admin] PATH_INFO: `/v3/projects` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:02.412084 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4fe722-b44a-457e-806a-de065aba531b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:19:02.414142 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4fe722-b44a-457e-806a-de065aba531b admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:19:02.441450 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1010/2040] 10.222.0.26 () {60 vars in 1152 bytes} [Thu Jul 14 09:19:02 2022] POST /identity/v3/projects => generated 389 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 09:19:02.446276 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ecdf71-f9c0-4a2b-a7c0-0c67a1877e92 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:19:02.457160 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ecdf71-f9c0-4a2b-a7c0-0c67a1877e92 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:19:02.526084 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336a32e2-ae29-486a-b60e-5a9f4374305d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:19:02.526803 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336a32e2-ae29-486a-b60e-5a9f4374305d admin admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:02.527089 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336a32e2-ae29-486a-b60e-5a9f4374305d admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:02.527400 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336a32e2-ae29-486a-b60e-5a9f4374305d admin admin] PATH_INFO: `/v3/users` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:02.528139 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336a32e2-ae29-486a-b60e-5a9f4374305d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:19:02.530103 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336a32e2-ae29-486a-b60e-5a9f4374305d admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:19:02.553973 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1031/2041] 10.222.0.26 () {60 vars in 1143 bytes} [Thu Jul 14 09:19:02 2022] POST /identity/v3/users => generated 295 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 09:19:02.558331 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4fe722-b44a-457e-806a-de065aba531b admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:19:02.568851 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4fe722-b44a-457e-806a-de065aba531b admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:19:02.573563 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2700cbd2-abcf-4fea-b8b2-b42d58f1c619 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:19:02.574236 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2700cbd2-abcf-4fea-b8b2-b42d58f1c619 admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:02.574521 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2700cbd2-abcf-4fea-b8b2-b42d58f1c619 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:02.574798 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2700cbd2-abcf-4fea-b8b2-b42d58f1c619 admin admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:02.575628 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2700cbd2-abcf-4fea-b8b2-b42d58f1c619 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:19:02.577237 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2700cbd2-abcf-4fea-b8b2-b42d58f1c619 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:19:02.588188 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1011/2042] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jul 14 09:19:02 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 09:19:02.592208 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336a32e2-ae29-486a-b60e-5a9f4374305d admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:19:02.602430 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336a32e2-ae29-486a-b60e-5a9f4374305d admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:19:02.607285 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34605dbb-c07c-4804-81b3-dff4f260a733 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:19:02.607982 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34605dbb-c07c-4804-81b3-dff4f260a733 admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:02.608287 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34605dbb-c07c-4804-81b3-dff4f260a733 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:02.608551 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34605dbb-c07c-4804-81b3-dff4f260a733 admin admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:02.609276 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34605dbb-c07c-4804-81b3-dff4f260a733 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:19:02.610914 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34605dbb-c07c-4804-81b3-dff4f260a733 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:19:02.623958 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1032/2043] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jul 14 09:19:02 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 09:19:02.628530 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2700cbd2-abcf-4fea-b8b2-b42d58f1c619 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:19:02.635929 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:19:02.639973 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2700cbd2-abcf-4fea-b8b2-b42d58f1c619 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:19:02.645771 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9376ed-a29c-4cd7-9130-9145b4b98808 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:19:02.646993 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9376ed-a29c-4cd7-9130-9145b4b98808 admin admin] REQUEST_METHOD: `PUT` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:02.647319 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9376ed-a29c-4cd7-9130-9145b4b98808 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:02.647606 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9376ed-a29c-4cd7-9130-9145b4b98808 admin admin] PATH_INFO: `/v3/projects/39418a3b1c424ed48cfc68d5ac048413/users/4afe1ec7d1ab4e40892d1dde44ce1667/roles/d1691322ce6a4edab7ce722edf0f2988` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:02.656466 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34605dbb-c07c-4804-81b3-dff4f260a733 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:19:02.658329 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-34605dbb-c07c-4804-81b3-dff4f260a733 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:19:02.666311 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9376ed-a29c-4cd7-9130-9145b4b98808 admin admin] RBAC: Authorizing `identity:create_grant(project_id=39418a3b1c424ed48cfc68d5ac048413, user_id=4afe1ec7d1ab4e40892d1dde44ce1667, role_id=d1691322ce6a4edab7ce722edf0f2988)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:19:02.669042 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9376ed-a29c-4cd7-9130-9145b4b98808 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:19:02.676345 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34605dbb-c07c-4804-81b3-dff4f260a733 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:19:02.682835 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65189074-e648-4b20-a18c-d9cc2c91f188 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:19:02.683804 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65189074-e648-4b20-a18c-d9cc2c91f188 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:02.684022 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65189074-e648-4b20-a18c-d9cc2c91f188 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:02.684353 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65189074-e648-4b20-a18c-d9cc2c91f188 admin admin] PATH_INFO: `/v3/projects/58be0432962d48429a67bf2656e43159` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:02.693912 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65189074-e648-4b20-a18c-d9cc2c91f188 admin admin] RBAC: Authorizing `identity:delete_project(project_id=58be0432962d48429a67bf2656e43159)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:19:02.693912 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65189074-e648-4b20-a18c-d9cc2c91f188 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:19:02.694150 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1012/2044] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jul 14 09:19:02 2022] PUT /identity/v3/projects/39418a3b1c424ed48cfc68d5ac048413/users/4afe1ec7d1ab4e40892d1dde44ce1667/roles/d1691322ce6a4edab7ce722edf0f2988 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:19:02.701487 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775a753b-3b78-446a-b693-b4f003eeff1e None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:02.701487 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775a753b-3b78-446a-b693-b4f003eeff1e None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:02.701487 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775a753b-3b78-446a-b693-b4f003eeff1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:02.716986 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-775a753b-3b78-446a-b693-b4f003eeff1e None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:19:02.718529 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-775a753b-3b78-446a-b693-b4f003eeff1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:19:02.737385 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-65189074-e648-4b20-a18c-d9cc2c91f188 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '58be0432962d48429a67bf2656e43159', 'request_id': 'req-65189074-e648-4b20-a18c-d9cc2c91f188'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:19:02.745206 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-65189074-e648-4b20-a18c-d9cc2c91f188 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '58be0432962d48429a67bf2656e43159', 'request_id': 'req-65189074-e648-4b20-a18c-d9cc2c91f188'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:19:02.752245 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-65189074-e648-4b20-a18c-d9cc2c91f188 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '58be0432962d48429a67bf2656e43159', 'request_id': 'req-65189074-e648-4b20-a18c-d9cc2c91f188'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:19:02.766785 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-65189074-e648-4b20-a18c-d9cc2c91f188 admin admin] The token cache is being invalidate because project 58be0432962d48429a67bf2656e43159 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80382) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 09:19:02.767071 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-65189074-e648-4b20-a18c-d9cc2c91f188 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:19:02.767754 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-65189074-e648-4b20-a18c-d9cc2c91f188 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:19:02.772719 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1033/2045] 10.222.0.26 () {58 vars in 1232 bytes} [Thu Jul 14 09:19:02 2022] DELETE /identity/v3/projects/58be0432962d48429a67bf2656e43159 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:19:02.772719 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d614359-7e56-4349-b508-f00192dfd236 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:02.772719 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d614359-7e56-4349-b508-f00192dfd236 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:02.772719 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d614359-7e56-4349-b508-f00192dfd236 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:02.780997 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1013/2046] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:19:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:19:02.781954 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775a753b-3b78-446a-b693-b4f003eeff1e None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:19:02.783433 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-775a753b-3b78-446a-b693-b4f003eeff1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:19:02.793150 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-7d614359-7e56-4349-b508-f00192dfd236 None None] MFA Rules not processed for user `38970dab4776492a8f2fccaf9705ebc0`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:19:02.794413 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-7d614359-7e56-4349-b508-f00192dfd236 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:19:02.795995 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775a753b-3b78-446a-b693-b4f003eeff1e None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:19:02.852542 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1034/2047] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:19:02 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:19:02.857244 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d614359-7e56-4349-b508-f00192dfd236 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:19:02.857244 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-7d614359-7e56-4349-b508-f00192dfd236 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:19:02.857385 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031e95c4-2b7c-435c-a398-18f542f8af43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:19:02.857385 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031e95c4-2b7c-435c-a398-18f542f8af43 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:02.857385 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031e95c4-2b7c-435c-a398-18f542f8af43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:02.857385 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031e95c4-2b7c-435c-a398-18f542f8af43 admin admin] PATH_INFO: `/v3/users/bae569df7d73402c9c2717932b088786` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:02.858441 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031e95c4-2b7c-435c-a398-18f542f8af43 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bae569df7d73402c9c2717932b088786)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:19:02.860107 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:19:02.860603 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031e95c4-2b7c-435c-a398-18f542f8af43 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:19:02.933265 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d614359-7e56-4349-b508-f00192dfd236 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:19:02.947282 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461b4808-fa5c-4ee5-a981-0720ec1b8dda service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '952911c492444ffea4b211a24b564117', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:19:02.949753 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461b4808-fa5c-4ee5-a981-0720ec1b8dda service neutron] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:02.950895 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461b4808-fa5c-4ee5-a981-0720ec1b8dda service neutron] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:02.951179 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461b4808-fa5c-4ee5-a981-0720ec1b8dda service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:02.954906 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-031e95c4-2b7c-435c-a398-18f542f8af43 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bae569df7d73402c9c2717932b088786'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:19:02.966079 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-031e95c4-2b7c-435c-a398-18f542f8af43 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bae569df7d73402c9c2717932b088786'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:19:02.967600 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-031e95c4-2b7c-435c-a398-18f542f8af43 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bae569df7d73402c9c2717932b088786'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:19:02.968614 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-031e95c4-2b7c-435c-a398-18f542f8af43 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bae569df7d73402c9c2717932b088786'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:19:02.969034 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461b4808-fa5c-4ee5-a981-0720ec1b8dda service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:19:02.970821 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461b4808-fa5c-4ee5-a981-0720ec1b8dda service neutron] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:19:02.978092 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-031e95c4-2b7c-435c-a398-18f542f8af43 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bae569df7d73402c9c2717932b088786'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:19:02.984977 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1035/2048] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:19:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:19:02.997426 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:19:03.002983 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1014/2049] 10.222.0.26 () {58 vars in 1223 bytes} [Thu Jul 14 09:19:02 2022] DELETE /identity/v3/users/bae569df7d73402c9c2717932b088786 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:19:03.537888 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461b4808-fa5c-4ee5-a981-0720ec1b8dda service neutron] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:19:03.551249 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461b4808-fa5c-4ee5-a981-0720ec1b8dda service neutron] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:19:03.661452 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243a5c28-69b0-4b54-b73a-049001c34d36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:19:03.662799 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243a5c28-69b0-4b54-b73a-049001c34d36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:03.663084 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243a5c28-69b0-4b54-b73a-049001c34d36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:03.663868 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243a5c28-69b0-4b54-b73a-049001c34d36 admin admin] PATH_INFO: `/v3/projects/f15eb397cd5342af8b3c1d5a685d245a` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:03.665588 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243a5c28-69b0-4b54-b73a-049001c34d36 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f15eb397cd5342af8b3c1d5a685d245a)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:19:03.667930 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243a5c28-69b0-4b54-b73a-049001c34d36 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:19:03.745307 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-243a5c28-69b0-4b54-b73a-049001c34d36 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f15eb397cd5342af8b3c1d5a685d245a', 'request_id': 'req-243a5c28-69b0-4b54-b73a-049001c34d36'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:19:03.756363 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-243a5c28-69b0-4b54-b73a-049001c34d36 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f15eb397cd5342af8b3c1d5a685d245a', 'request_id': 'req-243a5c28-69b0-4b54-b73a-049001c34d36'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:19:03.764706 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-243a5c28-69b0-4b54-b73a-049001c34d36 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f15eb397cd5342af8b3c1d5a685d245a', 'request_id': 'req-243a5c28-69b0-4b54-b73a-049001c34d36'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:19:03.783914 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-243a5c28-69b0-4b54-b73a-049001c34d36 admin admin] The token cache is being invalidate because project f15eb397cd5342af8b3c1d5a685d245a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80382) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 09:19:03.784680 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-243a5c28-69b0-4b54-b73a-049001c34d36 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:19:03.785795 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-243a5c28-69b0-4b54-b73a-049001c34d36 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:19:03.788836 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1036/2050] 10.222.0.26 () {58 vars in 1232 bytes} [Thu Jul 14 09:19:03 2022] DELETE /identity/v3/projects/f15eb397cd5342af8b3c1d5a685d245a => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:19:03.793801 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d82397c-9b85-42e5-ac5f-d07ce7f1cf7d None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:03.794670 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d82397c-9b85-42e5-ac5f-d07ce7f1cf7d None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:03.794670 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d82397c-9b85-42e5-ac5f-d07ce7f1cf7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:03.818351 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-0d82397c-9b85-42e5-ac5f-d07ce7f1cf7d None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:19:03.819933 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-0d82397c-9b85-42e5-ac5f-d07ce7f1cf7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:19:03.943733 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1015/2051] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:19:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:19:03.951782 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62b2851-0aff-460a-af4b-c9ad8b302809 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:03.952444 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62b2851-0aff-460a-af4b-c9ad8b302809 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:03.952444 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62b2851-0aff-460a-af4b-c9ad8b302809 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:03.992975 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-f62b2851-0aff-460a-af4b-c9ad8b302809 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:19:03.994795 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-f62b2851-0aff-460a-af4b-c9ad8b302809 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:19:04.022809 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1037/2052] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:19:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:19:04.024187 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d82397c-9b85-42e5-ac5f-d07ce7f1cf7d None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:19:04.039331 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d82397c-9b85-42e5-ac5f-d07ce7f1cf7d None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:19:04.046249 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc0a8c1-5896-431d-b995-20df6dc5c72e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:19:04.047170 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc0a8c1-5896-431d-b995-20df6dc5c72e admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:04.047528 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc0a8c1-5896-431d-b995-20df6dc5c72e admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:04.047922 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc0a8c1-5896-431d-b995-20df6dc5c72e admin admin] PATH_INFO: `/v3/domains` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:04.048749 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc0a8c1-5896-431d-b995-20df6dc5c72e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:19:04.049234 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc0a8c1-5896-431d-b995-20df6dc5c72e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:19:04.051057 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc0a8c1-5896-431d-b995-20df6dc5c72e admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:19:04.075948 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1016/2053] 10.222.0.26 () {58 vars in 1152 bytes} [Thu Jul 14 09:19:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:19:04.077787 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62b2851-0aff-460a-af4b-c9ad8b302809 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:19:04.089675 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62b2851-0aff-460a-af4b-c9ad8b302809 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:19:04.111525 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a37c10-9dbd-4899-8837-2ce184ce81bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:19:04.115866 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a37c10-9dbd-4899-8837-2ce184ce81bd admin admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:04.116494 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a37c10-9dbd-4899-8837-2ce184ce81bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:04.117678 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a37c10-9dbd-4899-8837-2ce184ce81bd admin admin] PATH_INFO: `/v3/projects` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:04.118852 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a37c10-9dbd-4899-8837-2ce184ce81bd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:19:04.123952 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a37c10-9dbd-4899-8837-2ce184ce81bd admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:19:04.173653 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1038/2054] 10.222.0.26 () {60 vars in 1152 bytes} [Thu Jul 14 09:19:04 2022] POST /identity/v3/projects => generated 385 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 09:19:04.174799 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc0a8c1-5896-431d-b995-20df6dc5c72e admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:19:04.189828 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc0a8c1-5896-431d-b995-20df6dc5c72e admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:19:04.287609 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3464ad29-fe35-48ad-8403-e54fdcb3cc00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:19:04.288264 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3464ad29-fe35-48ad-8403-e54fdcb3cc00 admin admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:04.288552 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3464ad29-fe35-48ad-8403-e54fdcb3cc00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:04.288825 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3464ad29-fe35-48ad-8403-e54fdcb3cc00 admin admin] PATH_INFO: `/v3/users` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:04.289521 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3464ad29-fe35-48ad-8403-e54fdcb3cc00 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:19:04.291489 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3464ad29-fe35-48ad-8403-e54fdcb3cc00 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:19:04.320096 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1017/2055] 10.222.0.26 () {60 vars in 1143 bytes} [Thu Jul 14 09:19:04 2022] POST /identity/v3/users => generated 293 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 09:19:04.328173 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a37c10-9dbd-4899-8837-2ce184ce81bd admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:19:04.351725 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a37c10-9dbd-4899-8837-2ce184ce81bd admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:19:04.358058 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9195c5f-8e4d-40ac-bbed-d91411758c54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:19:04.358750 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9195c5f-8e4d-40ac-bbed-d91411758c54 admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:04.359036 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9195c5f-8e4d-40ac-bbed-d91411758c54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:04.359373 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9195c5f-8e4d-40ac-bbed-d91411758c54 admin admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:04.360252 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9195c5f-8e4d-40ac-bbed-d91411758c54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:19:04.361945 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9195c5f-8e4d-40ac-bbed-d91411758c54 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:19:04.363960 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:19:04.375579 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1039/2056] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jul 14 09:19:04 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 09:19:04.379741 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3464ad29-fe35-48ad-8403-e54fdcb3cc00 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:19:04.404788 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3464ad29-fe35-48ad-8403-e54fdcb3cc00 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:19:04.414449 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe3d5db-9937-4d7e-880e-dd05105be0e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:19:04.415564 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe3d5db-9937-4d7e-880e-dd05105be0e3 admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:04.415860 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe3d5db-9937-4d7e-880e-dd05105be0e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:04.416423 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe3d5db-9937-4d7e-880e-dd05105be0e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:04.418501 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe3d5db-9937-4d7e-880e-dd05105be0e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:19:04.420971 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe3d5db-9937-4d7e-880e-dd05105be0e3 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:19:04.437896 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1018/2057] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jul 14 09:19:04 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 09:19:04.442155 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9195c5f-8e4d-40ac-bbed-d91411758c54 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:19:04.454606 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9195c5f-8e4d-40ac-bbed-d91411758c54 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:19:04.460973 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560e9d72-bbcb-4067-bdcb-6d29e28fda64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:19:04.462420 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560e9d72-bbcb-4067-bdcb-6d29e28fda64 admin admin] REQUEST_METHOD: `PUT` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:04.462420 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560e9d72-bbcb-4067-bdcb-6d29e28fda64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:04.462582 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560e9d72-bbcb-4067-bdcb-6d29e28fda64 admin admin] PATH_INFO: `/v3/projects/2d724787c4b3469db2a80ba744b51f7c/users/5b7f99f7e7aa48df9e84f5ce5506b867/roles/d1691322ce6a4edab7ce722edf0f2988` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:04.481365 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560e9d72-bbcb-4067-bdcb-6d29e28fda64 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d724787c4b3469db2a80ba744b51f7c, user_id=5b7f99f7e7aa48df9e84f5ce5506b867, role_id=d1691322ce6a4edab7ce722edf0f2988)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:19:04.484155 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560e9d72-bbcb-4067-bdcb-6d29e28fda64 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:19:04.504018 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1040/2058] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jul 14 09:19:04 2022] PUT /identity/v3/projects/2d724787c4b3469db2a80ba744b51f7c/users/5b7f99f7e7aa48df9e84f5ce5506b867/roles/d1691322ce6a4edab7ce722edf0f2988 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:19:04.520998 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18e74be-e5cc-4d53-af44-d39e0e83c820 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:04.526124 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18e74be-e5cc-4d53-af44-d39e0e83c820 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:04.526776 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18e74be-e5cc-4d53-af44-d39e0e83c820 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:04.575602 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-b18e74be-e5cc-4d53-af44-d39e0e83c820 None None] MFA Rules not processed for user `5b7f99f7e7aa48df9e84f5ce5506b867`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:19:04.577154 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-b18e74be-e5cc-4d53-af44-d39e0e83c820 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:19:04.646732 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:19:04.661911 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1019/2059] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:19:04 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:19:04.666205 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560e9d72-bbcb-4067-bdcb-6d29e28fda64 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:19:04.747509 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560e9d72-bbcb-4067-bdcb-6d29e28fda64 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:19:04.754585 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bab465-a81e-4b80-acfd-0987ca62a7eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:19:04.755312 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bab465-a81e-4b80-acfd-0987ca62a7eb admin admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:04.755598 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bab465-a81e-4b80-acfd-0987ca62a7eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:04.755875 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bab465-a81e-4b80-acfd-0987ca62a7eb admin admin] PATH_INFO: `/v3/projects` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:04.756616 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bab465-a81e-4b80-acfd-0987ca62a7eb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:19:04.758622 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bab465-a81e-4b80-acfd-0987ca62a7eb admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:19:04.791154 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1041/2060] 10.222.0.26 () {60 vars in 1152 bytes} [Thu Jul 14 09:19:04 2022] POST /identity/v3/projects => generated 385 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 09:19:04.795459 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18e74be-e5cc-4d53-af44-d39e0e83c820 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:19:04.823546 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18e74be-e5cc-4d53-af44-d39e0e83c820 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:19:04.933603 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d202240-5ec6-4422-a038-f3ff759b608a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:19:04.934711 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d202240-5ec6-4422-a038-f3ff759b608a admin admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:04.935928 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d202240-5ec6-4422-a038-f3ff759b608a admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:04.936406 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d202240-5ec6-4422-a038-f3ff759b608a admin admin] PATH_INFO: `/v3/users` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:04.937296 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d202240-5ec6-4422-a038-f3ff759b608a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:19:04.940636 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d202240-5ec6-4422-a038-f3ff759b608a admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:19:04.988399 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1020/2061] 10.222.0.26 () {60 vars in 1143 bytes} [Thu Jul 14 09:19:04 2022] POST /identity/v3/users => generated 299 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 09:19:04.988749 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bab465-a81e-4b80-acfd-0987ca62a7eb admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:19:05.009170 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bab465-a81e-4b80-acfd-0987ca62a7eb admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:19:05.018561 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fc05d9-18a6-47e4-a6e9-e4598041d5b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:19:05.019603 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fc05d9-18a6-47e4-a6e9-e4598041d5b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:05.020124 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fc05d9-18a6-47e4-a6e9-e4598041d5b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:05.020706 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fc05d9-18a6-47e4-a6e9-e4598041d5b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:05.022435 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fc05d9-18a6-47e4-a6e9-e4598041d5b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:19:05.026829 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fc05d9-18a6-47e4-a6e9-e4598041d5b9 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:19:05.052672 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1042/2062] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jul 14 09:19:04 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 09:19:05.056795 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d202240-5ec6-4422-a038-f3ff759b608a admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:19:05.068400 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d202240-5ec6-4422-a038-f3ff759b608a admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:19:05.083340 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922bc4a3-fda7-44ee-9b72-7ed6822722c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:19:05.084637 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922bc4a3-fda7-44ee-9b72-7ed6822722c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:05.085093 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922bc4a3-fda7-44ee-9b72-7ed6822722c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:05.085519 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922bc4a3-fda7-44ee-9b72-7ed6822722c3 admin admin] PATH_INFO: `/v3/projects/7584e47cf97e46768eacf7900991bb2d/users/0a42f772e8e84e6b8f7e5ce67bb9009d/roles/7b549d58d1724395b343609d895c80d0` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:05.107534 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922bc4a3-fda7-44ee-9b72-7ed6822722c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7584e47cf97e46768eacf7900991bb2d, user_id=0a42f772e8e84e6b8f7e5ce67bb9009d, role_id=7b549d58d1724395b343609d895c80d0)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:19:05.113461 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922bc4a3-fda7-44ee-9b72-7ed6822722c3 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:19:05.133877 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1021/2063] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jul 14 09:19:05 2022] PUT /identity/v3/projects/7584e47cf97e46768eacf7900991bb2d/users/0a42f772e8e84e6b8f7e5ce67bb9009d/roles/7b549d58d1724395b343609d895c80d0 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:19:05.138011 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fc05d9-18a6-47e4-a6e9-e4598041d5b9 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:19:05.151942 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fc05d9-18a6-47e4-a6e9-e4598041d5b9 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:19:05.283829 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6185d5e-9d68-4abd-af3e-e78f0ee730d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:19:05.284525 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6185d5e-9d68-4abd-af3e-e78f0ee730d1 admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:05.284806 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6185d5e-9d68-4abd-af3e-e78f0ee730d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:05.285081 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6185d5e-9d68-4abd-af3e-e78f0ee730d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:05.285827 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6185d5e-9d68-4abd-af3e-e78f0ee730d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:19:05.290542 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6185d5e-9d68-4abd-af3e-e78f0ee730d1 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:19:05.302932 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1043/2064] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jul 14 09:19:05 2022] GET /identity/v3/roles => generated 1420 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 09:19:05.307125 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922bc4a3-fda7-44ee-9b72-7ed6822722c3 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:19:05.318765 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922bc4a3-fda7-44ee-9b72-7ed6822722c3 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:19:05.324946 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b844b39-194c-46aa-a7c3-69f333d0168b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:19:05.326269 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b844b39-194c-46aa-a7c3-69f333d0168b admin admin] REQUEST_METHOD: `PUT` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:05.326763 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b844b39-194c-46aa-a7c3-69f333d0168b admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:05.327157 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b844b39-194c-46aa-a7c3-69f333d0168b admin admin] PATH_INFO: `/v3/projects/7584e47cf97e46768eacf7900991bb2d/users/0a42f772e8e84e6b8f7e5ce67bb9009d/roles/7b549d58d1724395b343609d895c80d0` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:05.329413 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b844b39-194c-46aa-a7c3-69f333d0168b admin admin] RBAC: Authorizing `identity:create_grant(project_id=7584e47cf97e46768eacf7900991bb2d, user_id=0a42f772e8e84e6b8f7e5ce67bb9009d, role_id=7b549d58d1724395b343609d895c80d0)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:19:05.332500 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b844b39-194c-46aa-a7c3-69f333d0168b admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:19:05.346169 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1022/2065] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jul 14 09:19:05 2022] PUT /identity/v3/projects/7584e47cf97e46768eacf7900991bb2d/users/0a42f772e8e84e6b8f7e5ce67bb9009d/roles/7b549d58d1724395b343609d895c80d0 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 09:19:05.358691 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8af5d6-090d-499c-b586-a04610999335 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:05.358988 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8af5d6-090d-499c-b586-a04610999335 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:05.359310 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8af5d6-090d-499c-b586-a04610999335 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:05.395351 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-7c8af5d6-090d-499c-b586-a04610999335 None None] MFA Rules not processed for user `0a42f772e8e84e6b8f7e5ce67bb9009d`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:19:05.396794 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-7c8af5d6-090d-499c-b586-a04610999335 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:19:05.464531 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:19:05.520122 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1044/2066] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:19:05 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:19:05.543571 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b844b39-194c-46aa-a7c3-69f333d0168b admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:19:05.545599 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-0b844b39-194c-46aa-a7c3-69f333d0168b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:19:05.685386 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b844b39-194c-46aa-a7c3-69f333d0168b admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:19:05.696869 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023a3195-17bd-4c5f-9e87-cc6934c6c8e3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e76f1952a5b146808844b9f2b7395035', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:19:05.697597 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023a3195-17bd-4c5f-9e87-cc6934c6c8e3 service cinder] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:05.697897 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023a3195-17bd-4c5f-9e87-cc6934c6c8e3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:05.698158 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023a3195-17bd-4c5f-9e87-cc6934c6c8e3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:05.818331 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023a3195-17bd-4c5f-9e87-cc6934c6c8e3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:19:05.820133 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023a3195-17bd-4c5f-9e87-cc6934c6c8e3 service cinder] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:19:05.826072 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:19:05.834520 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1023/2067] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:19:05 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:19:09.683749 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8af5d6-090d-499c-b586-a04610999335 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:19:09.686232 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-7c8af5d6-090d-499c-b586-a04610999335 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:19:09.763930 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8af5d6-090d-499c-b586-a04610999335 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:19:09.768350 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4bfc7f-6f5e-42eb-82ae-4a0ab78bdf6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:19:09.769233 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4bfc7f-6f5e-42eb-82ae-4a0ab78bdf6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:09.769516 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4bfc7f-6f5e-42eb-82ae-4a0ab78bdf6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:09.769786 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4bfc7f-6f5e-42eb-82ae-4a0ab78bdf6c admin admin] PATH_INFO: `/v3/users/23e0aaea1ff24fdfbfe23cdfbdb5ffe3` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:09.770977 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4bfc7f-6f5e-42eb-82ae-4a0ab78bdf6c admin admin] RBAC: Authorizing `identity:delete_user(user_id=23e0aaea1ff24fdfbfe23cdfbdb5ffe3)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:19:09.773225 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4bfc7f-6f5e-42eb-82ae-4a0ab78bdf6c admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:19:09.789352 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604c7949-0243-4e74-809b-428e8e69bab8 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:09.789847 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604c7949-0243-4e74-809b-428e8e69bab8 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:09.789951 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604c7949-0243-4e74-809b-428e8e69bab8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:09.820137 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-604c7949-0243-4e74-809b-428e8e69bab8 None None] MFA Rules not processed for user `4afe1ec7d1ab4e40892d1dde44ce1667`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:19:09.822133 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-604c7949-0243-4e74-809b-428e8e69bab8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:19:09.832671 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-0c4bfc7f-6f5e-42eb-82ae-4a0ab78bdf6c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '23e0aaea1ff24fdfbfe23cdfbdb5ffe3'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:19:09.833416 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-0c4bfc7f-6f5e-42eb-82ae-4a0ab78bdf6c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '23e0aaea1ff24fdfbfe23cdfbdb5ffe3'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:19:09.849523 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-0c4bfc7f-6f5e-42eb-82ae-4a0ab78bdf6c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '23e0aaea1ff24fdfbfe23cdfbdb5ffe3'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:19:09.859688 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-0c4bfc7f-6f5e-42eb-82ae-4a0ab78bdf6c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '23e0aaea1ff24fdfbfe23cdfbdb5ffe3'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:19:09.860376 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-0c4bfc7f-6f5e-42eb-82ae-4a0ab78bdf6c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '23e0aaea1ff24fdfbfe23cdfbdb5ffe3'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:19:09.876625 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1045/2068] 10.222.0.26 () {58 vars in 1223 bytes} [Thu Jul 14 09:19:09 2022] DELETE /identity/v3/users/23e0aaea1ff24fdfbfe23cdfbdb5ffe3 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:19:09.880975 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6985713a-6cfa-4673-b021-f40796e84041 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:09.881400 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6985713a-6cfa-4673-b021-f40796e84041 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:09.881736 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6985713a-6cfa-4673-b021-f40796e84041 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:09.901511 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-6985713a-6cfa-4673-b021-f40796e84041 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:19:09.902759 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-6985713a-6cfa-4673-b021-f40796e84041 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:19:09.913361 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1024/2069] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:19:09 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 09:19:09.926980 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604c7949-0243-4e74-809b-428e8e69bab8 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:19:09.928399 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-604c7949-0243-4e74-809b-428e8e69bab8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:19:09.964260 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:19:09.992120 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1046/2070] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:19:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 09:19:10.003430 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6985713a-6cfa-4673-b021-f40796e84041 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:19:10.005057 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-6985713a-6cfa-4673-b021-f40796e84041 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:19:10.026762 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604c7949-0243-4e74-809b-428e8e69bab8 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:19:10.109007 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6985713a-6cfa-4673-b021-f40796e84041 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:19:10.114038 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f61cee0-78d1-43ab-b4c8-db4227e2e150 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '952911c492444ffea4b211a24b564117', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:19:10.116290 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f61cee0-78d1-43ab-b4c8-db4227e2e150 service neutron] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:10.116290 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f61cee0-78d1-43ab-b4c8-db4227e2e150 service neutron] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:10.116290 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f61cee0-78d1-43ab-b4c8-db4227e2e150 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:10.132288 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f61cee0-78d1-43ab-b4c8-db4227e2e150 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:19:10.133779 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f61cee0-78d1-43ab-b4c8-db4227e2e150 service neutron] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:19:10.149746 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1047/2071] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:19:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 09:19:10.159388 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87732509-e017-48cb-bf9a-abfc4a9c1266 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '538fff3c5a2b487eb44ae7c1a1198102', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:19:10.160349 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87732509-e017-48cb-bf9a-abfc4a9c1266 service nova] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:10.160706 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87732509-e017-48cb-bf9a-abfc4a9c1266 service nova] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:10.161047 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87732509-e017-48cb-bf9a-abfc4a9c1266 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:10.195609 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:19:10.272303 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87732509-e017-48cb-bf9a-abfc4a9c1266 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:19:10.274054 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87732509-e017-48cb-bf9a-abfc4a9c1266 service nova] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:19:10.285129 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1025/2072] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:19:09 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:19:10.597819 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f61cee0-78d1-43ab-b4c8-db4227e2e150 service neutron] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:19:10.599303 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-7f61cee0-78d1-43ab-b4c8-db4227e2e150 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:19:10.632011 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87732509-e017-48cb-bf9a-abfc4a9c1266 service nova] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:19:10.633828 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-87732509-e017-48cb-bf9a-abfc4a9c1266 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:19:10.682905 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87732509-e017-48cb-bf9a-abfc4a9c1266 service nova] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:19:10.705174 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc257eb-22f3-415c-a621-8d5485124ede admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:19:10.706092 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc257eb-22f3-415c-a621-8d5485124ede admin admin] REQUEST_METHOD: `DELETE` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:10.706377 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc257eb-22f3-415c-a621-8d5485124ede admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:10.706651 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc257eb-22f3-415c-a621-8d5485124ede admin admin] PATH_INFO: `/v3/projects/d4f7361e8c32486eaf05bb15f240ba51` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:10.710662 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc257eb-22f3-415c-a621-8d5485124ede admin admin] RBAC: Authorizing `identity:delete_project(project_id=d4f7361e8c32486eaf05bb15f240ba51)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:19:10.712642 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc257eb-22f3-415c-a621-8d5485124ede admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:19:10.754838 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f61cee0-78d1-43ab-b4c8-db4227e2e150 service neutron] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:19:10.771996 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060ade78-f46c-4391-991f-412c537203d3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e76f1952a5b146808844b9f2b7395035', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:19:10.773003 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060ade78-f46c-4391-991f-412c537203d3 service cinder] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:10.773411 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060ade78-f46c-4391-991f-412c537203d3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:10.773780 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060ade78-f46c-4391-991f-412c537203d3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:10.776294 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-060ade78-f46c-4391-991f-412c537203d3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:19:10.806791 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-9cc257eb-22f3-415c-a621-8d5485124ede admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd4f7361e8c32486eaf05bb15f240ba51', 'request_id': 'req-9cc257eb-22f3-415c-a621-8d5485124ede'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:19:10.818297 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-9cc257eb-22f3-415c-a621-8d5485124ede admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd4f7361e8c32486eaf05bb15f240ba51', 'request_id': 'req-9cc257eb-22f3-415c-a621-8d5485124ede'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:19:10.828783 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-9cc257eb-22f3-415c-a621-8d5485124ede admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd4f7361e8c32486eaf05bb15f240ba51', 'request_id': 'req-9cc257eb-22f3-415c-a621-8d5485124ede'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:19:10.850520 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-9cc257eb-22f3-415c-a621-8d5485124ede admin admin] The token cache is being invalidate because project d4f7361e8c32486eaf05bb15f240ba51 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80381) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 09:19:10.851096 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-9cc257eb-22f3-415c-a621-8d5485124ede admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:19:10.852638 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-9cc257eb-22f3-415c-a621-8d5485124ede admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:19:10.858125 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1026/2073] 10.222.0.26 () {58 vars in 1232 bytes} [Thu Jul 14 09:19:10 2022] DELETE /identity/v3/projects/d4f7361e8c32486eaf05bb15f240ba51 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:19:10.860815 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a5a547-9418-48b2-b122-3f3bc98ba280 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:10.861262 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a5a547-9418-48b2-b122-3f3bc98ba280 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:10.861553 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a5a547-9418-48b2-b122-3f3bc98ba280 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:10.864117 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060ade78-f46c-4391-991f-412c537203d3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:19:10.865931 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060ade78-f46c-4391-991f-412c537203d3 service cinder] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:19:10.878495 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1048/2074] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:19:10 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 281 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:19:10.887455 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-95a5a547-9418-48b2-b122-3f3bc98ba280 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:19:10.888734 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-95a5a547-9418-48b2-b122-3f3bc98ba280 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:19:10.996687 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1027/2075] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:19:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:19:10.999106 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2eb9a7-e533-4bd6-9f3d-ec71e43425d4 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:10.999443 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2eb9a7-e533-4bd6-9f3d-ec71e43425d4 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:11.000274 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2eb9a7-e533-4bd6-9f3d-ec71e43425d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:11.046168 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-bf2eb9a7-e533-4bd6-9f3d-ec71e43425d4 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:19:11.048247 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-bf2eb9a7-e533-4bd6-9f3d-ec71e43425d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:19:11.072228 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1049/2076] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:19:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 09:19:11.077650 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a5a547-9418-48b2-b122-3f3bc98ba280 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:19:11.096164 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a5a547-9418-48b2-b122-3f3bc98ba280 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:19:11.112067 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759577f0-7c01-438e-8f69-612e3c50e358 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:19:11.114215 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759577f0-7c01-438e-8f69-612e3c50e358 admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:11.115156 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759577f0-7c01-438e-8f69-612e3c50e358 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:11.115839 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759577f0-7c01-438e-8f69-612e3c50e358 admin admin] PATH_INFO: `/v3/domains` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:11.119760 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759577f0-7c01-438e-8f69-612e3c50e358 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:19:11.119760 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759577f0-7c01-438e-8f69-612e3c50e358 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:19:11.121008 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759577f0-7c01-438e-8f69-612e3c50e358 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:19:11.162080 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1028/2077] 10.222.0.26 () {58 vars in 1152 bytes} [Thu Jul 14 09:19:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:19:11.164084 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2eb9a7-e533-4bd6-9f3d-ec71e43425d4 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:19:11.180875 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2eb9a7-e533-4bd6-9f3d-ec71e43425d4 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:19:11.198325 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9986935-e9cc-441d-bdb8-ed6ee64f25cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:19:11.199154 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9986935-e9cc-441d-bdb8-ed6ee64f25cf admin admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:11.199882 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9986935-e9cc-441d-bdb8-ed6ee64f25cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:11.200230 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9986935-e9cc-441d-bdb8-ed6ee64f25cf admin admin] PATH_INFO: `/v3/projects` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:11.201143 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9986935-e9cc-441d-bdb8-ed6ee64f25cf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:19:11.203490 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9986935-e9cc-441d-bdb8-ed6ee64f25cf admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:19:11.244659 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1050/2078] 10.222.0.26 () {60 vars in 1152 bytes} [Thu Jul 14 09:19:11 2022] POST /identity/v3/projects => generated 385 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 09:19:11.245702 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759577f0-7c01-438e-8f69-612e3c50e358 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:19:11.263018 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759577f0-7c01-438e-8f69-612e3c50e358 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:19:11.353309 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:19:11.404351 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0264a59-1025-4173-9897-766cc7f69f71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:19:11.405284 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0264a59-1025-4173-9897-766cc7f69f71 admin admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:11.405400 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0264a59-1025-4173-9897-766cc7f69f71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:11.405682 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0264a59-1025-4173-9897-766cc7f69f71 admin admin] PATH_INFO: `/v3/users` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:11.408007 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0264a59-1025-4173-9897-766cc7f69f71 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:19:11.411102 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0264a59-1025-4173-9897-766cc7f69f71 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:19:11.443507 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1029/2079] 10.222.0.26 () {60 vars in 1143 bytes} [Thu Jul 14 09:19:11 2022] POST /identity/v3/users => generated 293 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 09:19:11.447711 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9986935-e9cc-441d-bdb8-ed6ee64f25cf admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:19:11.460129 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9986935-e9cc-441d-bdb8-ed6ee64f25cf admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:19:11.470590 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a953c9a-4753-4691-947d-6c02387bb800 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:19:11.471478 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a953c9a-4753-4691-947d-6c02387bb800 admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:11.477415 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a953c9a-4753-4691-947d-6c02387bb800 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:11.477770 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a953c9a-4753-4691-947d-6c02387bb800 admin admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:11.478827 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a953c9a-4753-4691-947d-6c02387bb800 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:19:11.480849 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a953c9a-4753-4691-947d-6c02387bb800 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:19:11.495653 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:19:11.506487 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1051/2080] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jul 14 09:19:11 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 09:19:11.511486 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0264a59-1025-4173-9897-766cc7f69f71 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:19:11.526843 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0264a59-1025-4173-9897-766cc7f69f71 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:19:11.545191 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfc8fc8-64dd-4ff3-b23b-bef949db2618 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:19:11.546014 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfc8fc8-64dd-4ff3-b23b-bef949db2618 admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:11.546488 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfc8fc8-64dd-4ff3-b23b-bef949db2618 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:11.552845 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfc8fc8-64dd-4ff3-b23b-bef949db2618 admin admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:11.553750 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfc8fc8-64dd-4ff3-b23b-bef949db2618 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:19:11.555416 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfc8fc8-64dd-4ff3-b23b-bef949db2618 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:19:11.570283 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1030/2081] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jul 14 09:19:11 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 09:19:11.571702 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a953c9a-4753-4691-947d-6c02387bb800 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:19:11.583745 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a953c9a-4753-4691-947d-6c02387bb800 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:19:11.590418 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc650a7-eabb-4dff-bba4-04bcc3808638 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:19:11.592229 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc650a7-eabb-4dff-bba4-04bcc3808638 admin admin] REQUEST_METHOD: `PUT` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:11.592555 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc650a7-eabb-4dff-bba4-04bcc3808638 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:11.592975 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc650a7-eabb-4dff-bba4-04bcc3808638 admin admin] PATH_INFO: `/v3/projects/349f91a94f264213a1e9fdcf847e93a4/users/de74ccad93f742e6b43e4c4cd4550986/roles/d1691322ce6a4edab7ce722edf0f2988` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:11.614448 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc650a7-eabb-4dff-bba4-04bcc3808638 admin admin] RBAC: Authorizing `identity:create_grant(project_id=349f91a94f264213a1e9fdcf847e93a4, user_id=de74ccad93f742e6b43e4c4cd4550986, role_id=d1691322ce6a4edab7ce722edf0f2988)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:19:11.617213 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc650a7-eabb-4dff-bba4-04bcc3808638 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:19:11.642906 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1052/2082] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jul 14 09:19:11 2022] PUT /identity/v3/projects/349f91a94f264213a1e9fdcf847e93a4/users/de74ccad93f742e6b43e4c4cd4550986/roles/d1691322ce6a4edab7ce722edf0f2988 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:19:11.654035 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e235dd-f6e3-4b81-afcf-1a14a42c3438 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:11.654422 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e235dd-f6e3-4b81-afcf-1a14a42c3438 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:11.654609 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e235dd-f6e3-4b81-afcf-1a14a42c3438 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:11.687105 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-95e235dd-f6e3-4b81-afcf-1a14a42c3438 None None] MFA Rules not processed for user `de74ccad93f742e6b43e4c4cd4550986`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:19:11.690016 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-95e235dd-f6e3-4b81-afcf-1a14a42c3438 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:19:11.814428 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1031/2083] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:19:11 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:19:11.815412 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc650a7-eabb-4dff-bba4-04bcc3808638 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:19:11.939842 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc650a7-eabb-4dff-bba4-04bcc3808638 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:19:11.945528 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28aab93-f434-4337-bea2-29730af2b070 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:19:11.946396 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28aab93-f434-4337-bea2-29730af2b070 admin admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:11.946870 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28aab93-f434-4337-bea2-29730af2b070 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:11.947271 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28aab93-f434-4337-bea2-29730af2b070 admin admin] PATH_INFO: `/v3/projects` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:11.948254 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28aab93-f434-4337-bea2-29730af2b070 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:19:11.950708 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28aab93-f434-4337-bea2-29730af2b070 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:19:11.992482 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1053/2084] 10.222.0.26 () {60 vars in 1152 bytes} [Thu Jul 14 09:19:11 2022] POST /identity/v3/projects => generated 385 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 09:19:11.996372 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e235dd-f6e3-4b81-afcf-1a14a42c3438 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:19:12.013111 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e235dd-f6e3-4b81-afcf-1a14a42c3438 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:19:12.123869 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe4796e-612f-465f-93bc-1cc96460cb3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:19:12.125231 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe4796e-612f-465f-93bc-1cc96460cb3e admin admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:12.125690 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe4796e-612f-465f-93bc-1cc96460cb3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:12.126029 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe4796e-612f-465f-93bc-1cc96460cb3e admin admin] PATH_INFO: `/v3/users` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:12.127338 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe4796e-612f-465f-93bc-1cc96460cb3e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:19:12.129675 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe4796e-612f-465f-93bc-1cc96460cb3e admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:19:12.169798 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1032/2085] 10.222.0.26 () {60 vars in 1143 bytes} [Thu Jul 14 09:19:11 2022] POST /identity/v3/users => generated 293 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 09:19:12.174878 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28aab93-f434-4337-bea2-29730af2b070 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:19:12.191122 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28aab93-f434-4337-bea2-29730af2b070 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:19:12.197482 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6d8446-8fbc-4f82-b230-5a5ba5ad5b4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:19:12.198184 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6d8446-8fbc-4f82-b230-5a5ba5ad5b4c admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:12.198475 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6d8446-8fbc-4f82-b230-5a5ba5ad5b4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:12.198894 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6d8446-8fbc-4f82-b230-5a5ba5ad5b4c admin admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:12.199537 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6d8446-8fbc-4f82-b230-5a5ba5ad5b4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:19:12.201583 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6d8446-8fbc-4f82-b230-5a5ba5ad5b4c admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:19:12.219915 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe4796e-612f-465f-93bc-1cc96460cb3e admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:19:12.226473 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1054/2086] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jul 14 09:19:12 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 09:19:12.234856 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe4796e-612f-465f-93bc-1cc96460cb3e admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:19:12.255449 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1df237-8c39-49c0-8b7f-69f6be98a189 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:19:12.256260 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1df237-8c39-49c0-8b7f-69f6be98a189 admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:12.256425 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1df237-8c39-49c0-8b7f-69f6be98a189 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:12.256746 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1df237-8c39-49c0-8b7f-69f6be98a189 admin admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:12.257573 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1df237-8c39-49c0-8b7f-69f6be98a189 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:19:12.259282 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1df237-8c39-49c0-8b7f-69f6be98a189 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:19:12.277882 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6d8446-8fbc-4f82-b230-5a5ba5ad5b4c admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:19:12.278459 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1033/2087] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jul 14 09:19:12 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 09:19:12.289924 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6d8446-8fbc-4f82-b230-5a5ba5ad5b4c admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:19:12.299898 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a849a592-3605-4599-bd56-20047bbfc4f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:19:12.301569 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a849a592-3605-4599-bd56-20047bbfc4f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:12.301569 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a849a592-3605-4599-bd56-20047bbfc4f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:12.301877 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a849a592-3605-4599-bd56-20047bbfc4f5 admin admin] PATH_INFO: `/v3/projects/3a4bd179449c4c4d85386f4ba069c775/users/7bd7ab75c8bc4f4caf0383f22246e2ed/roles/d1691322ce6a4edab7ce722edf0f2988` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:12.319135 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a849a592-3605-4599-bd56-20047bbfc4f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a4bd179449c4c4d85386f4ba069c775, user_id=7bd7ab75c8bc4f4caf0383f22246e2ed, role_id=d1691322ce6a4edab7ce722edf0f2988)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:19:12.322308 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a849a592-3605-4599-bd56-20047bbfc4f5 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:19:12.364406 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db91bb3e-9e73-4eb2-8724-90bb53bd4d3f None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:12.364871 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db91bb3e-9e73-4eb2-8724-90bb53bd4d3f None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:12.365235 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db91bb3e-9e73-4eb2-8724-90bb53bd4d3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:12.368160 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1055/2088] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jul 14 09:19:12 2022] PUT /identity/v3/projects/3a4bd179449c4c4d85386f4ba069c775/users/7bd7ab75c8bc4f4caf0383f22246e2ed/roles/d1691322ce6a4edab7ce722edf0f2988 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:19:12.410596 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-db91bb3e-9e73-4eb2-8724-90bb53bd4d3f None None] MFA Rules not processed for user `7bd7ab75c8bc4f4caf0383f22246e2ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:19:12.415859 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-db91bb3e-9e73-4eb2-8724-90bb53bd4d3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:19:12.436924 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:19:12.539882 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1034/2089] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:19:12 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:19:12.544144 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a849a592-3605-4599-bd56-20047bbfc4f5 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:19:12.619489 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a849a592-3605-4599-bd56-20047bbfc4f5 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:19:12.624727 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b0be97-6c62-4ddc-a2a5-1e7aeb63e6a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:19:12.625388 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b0be97-6c62-4ddc-a2a5-1e7aeb63e6a4 admin admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:12.625685 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b0be97-6c62-4ddc-a2a5-1e7aeb63e6a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:12.625905 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b0be97-6c62-4ddc-a2a5-1e7aeb63e6a4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:12.626602 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b0be97-6c62-4ddc-a2a5-1e7aeb63e6a4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:19:12.629122 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b0be97-6c62-4ddc-a2a5-1e7aeb63e6a4 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:19:12.666544 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1056/2090] 10.222.0.26 () {60 vars in 1152 bytes} [Thu Jul 14 09:19:12 2022] POST /identity/v3/projects => generated 385 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 09:19:12.668579 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db91bb3e-9e73-4eb2-8724-90bb53bd4d3f None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:19:12.681798 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db91bb3e-9e73-4eb2-8724-90bb53bd4d3f None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:19:12.779985 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b1757f-7cc2-4540-b4e9-7f7c9efe4eb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:19:12.780667 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b1757f-7cc2-4540-b4e9-7f7c9efe4eb7 admin admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:12.780991 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b1757f-7cc2-4540-b4e9-7f7c9efe4eb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:12.781273 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b1757f-7cc2-4540-b4e9-7f7c9efe4eb7 admin admin] PATH_INFO: `/v3/users` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:12.782078 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b1757f-7cc2-4540-b4e9-7f7c9efe4eb7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:19:12.783924 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b1757f-7cc2-4540-b4e9-7f7c9efe4eb7 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:19:12.820589 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1035/2091] 10.222.0.26 () {60 vars in 1143 bytes} [Thu Jul 14 09:19:12 2022] POST /identity/v3/users => generated 299 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 09:19:12.822081 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b0be97-6c62-4ddc-a2a5-1e7aeb63e6a4 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:19:12.833091 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b0be97-6c62-4ddc-a2a5-1e7aeb63e6a4 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:19:12.840202 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724cdff0-5496-4774-976d-a6cf62fa8c68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:19:12.841102 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724cdff0-5496-4774-976d-a6cf62fa8c68 admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:12.842075 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724cdff0-5496-4774-976d-a6cf62fa8c68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:12.842894 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724cdff0-5496-4774-976d-a6cf62fa8c68 admin admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:12.843988 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724cdff0-5496-4774-976d-a6cf62fa8c68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:19:12.845675 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724cdff0-5496-4774-976d-a6cf62fa8c68 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:19:12.855019 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:19:12.872265 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543b61ae-d9f9-46ee-88d4-ec3b113b06fd None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:12.872915 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543b61ae-d9f9-46ee-88d4-ec3b113b06fd None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:12.873326 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543b61ae-d9f9-46ee-88d4-ec3b113b06fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:12.878172 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1057/2092] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jul 14 09:19:12 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 09:19:12.885089 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724cdff0-5496-4774-976d-a6cf62fa8c68 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:19:12.900084 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724cdff0-5496-4774-976d-a6cf62fa8c68 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:19:12.905713 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5caec83-f7cb-4348-aa41-08cee8db37a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:19:12.906810 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5caec83-f7cb-4348-aa41-08cee8db37a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:12.907034 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5caec83-f7cb-4348-aa41-08cee8db37a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:12.907472 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5caec83-f7cb-4348-aa41-08cee8db37a4 admin admin] PATH_INFO: `/v3/projects/a2e13ff34a3549f4b34509833e5f073e/users/0a3d2ce3e6d14de184e51d13043aa4e0/roles/7b549d58d1724395b343609d895c80d0` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:12.917031 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-543b61ae-d9f9-46ee-88d4-ec3b113b06fd None None] MFA Rules not processed for user `4afe1ec7d1ab4e40892d1dde44ce1667`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:19:12.920338 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-543b61ae-d9f9-46ee-88d4-ec3b113b06fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:19:12.925234 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5caec83-f7cb-4348-aa41-08cee8db37a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a2e13ff34a3549f4b34509833e5f073e, user_id=0a3d2ce3e6d14de184e51d13043aa4e0, role_id=7b549d58d1724395b343609d895c80d0)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:19:12.928261 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5caec83-f7cb-4348-aa41-08cee8db37a4 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:19:12.954944 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1058/2093] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jul 14 09:19:12 2022] PUT /identity/v3/projects/a2e13ff34a3549f4b34509833e5f073e/users/0a3d2ce3e6d14de184e51d13043aa4e0/roles/7b549d58d1724395b343609d895c80d0 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:19:12.958843 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5caec83-f7cb-4348-aa41-08cee8db37a4 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:19:12.991568 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1036/2094] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:19:12 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 09:19:13.061940 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5caec83-f7cb-4348-aa41-08cee8db37a4 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:19:13.067516 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3121e68c-1ea5-4c6b-b824-f13632ff8531 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:19:13.068288 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3121e68c-1ea5-4c6b-b824-f13632ff8531 admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:13.068634 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3121e68c-1ea5-4c6b-b824-f13632ff8531 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:13.068979 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3121e68c-1ea5-4c6b-b824-f13632ff8531 admin admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:13.069805 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3121e68c-1ea5-4c6b-b824-f13632ff8531 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:19:13.071530 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3121e68c-1ea5-4c6b-b824-f13632ff8531 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:19:13.085762 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1059/2095] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jul 14 09:19:12 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 09:19:13.089740 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543b61ae-d9f9-46ee-88d4-ec3b113b06fd None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:19:13.114824 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543b61ae-d9f9-46ee-88d4-ec3b113b06fd None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:19:13.120344 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e67d24-a7e0-4548-88b9-32c6444cd03f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:19:13.121405 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e67d24-a7e0-4548-88b9-32c6444cd03f admin admin] REQUEST_METHOD: `PUT` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:13.121756 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e67d24-a7e0-4548-88b9-32c6444cd03f admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:13.122129 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e67d24-a7e0-4548-88b9-32c6444cd03f admin admin] PATH_INFO: `/v3/projects/a2e13ff34a3549f4b34509833e5f073e/users/0a3d2ce3e6d14de184e51d13043aa4e0/roles/7b549d58d1724395b343609d895c80d0` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:13.124806 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e67d24-a7e0-4548-88b9-32c6444cd03f admin admin] RBAC: Authorizing `identity:create_grant(project_id=a2e13ff34a3549f4b34509833e5f073e, user_id=0a3d2ce3e6d14de184e51d13043aa4e0, role_id=7b549d58d1724395b343609d895c80d0)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:19:13.131509 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e67d24-a7e0-4548-88b9-32c6444cd03f admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:19:13.151744 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1037/2096] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jul 14 09:19:13 2022] PUT /identity/v3/projects/a2e13ff34a3549f4b34509833e5f073e/users/0a3d2ce3e6d14de184e51d13043aa4e0/roles/7b549d58d1724395b343609d895c80d0 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:19:13.159897 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5674793-4fd2-4b3f-8c16-50f776965ac7 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:13.160256 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5674793-4fd2-4b3f-8c16-50f776965ac7 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:13.160616 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5674793-4fd2-4b3f-8c16-50f776965ac7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:13.210124 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-f5674793-4fd2-4b3f-8c16-50f776965ac7 None None] MFA Rules not processed for user `0a3d2ce3e6d14de184e51d13043aa4e0`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:19:13.211569 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-f5674793-4fd2-4b3f-8c16-50f776965ac7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:19:13.308102 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1060/2097] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:19:13 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:19:13.320097 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e67d24-a7e0-4548-88b9-32c6444cd03f admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:19:13.322360 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-60e67d24-a7e0-4548-88b9-32c6444cd03f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:19:13.381060 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:19:13.413057 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e67d24-a7e0-4548-88b9-32c6444cd03f admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:19:13.417455 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6245df9-41f0-4cdc-9887-33c0e922f99d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e76f1952a5b146808844b9f2b7395035', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:19:13.418622 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6245df9-41f0-4cdc-9887-33c0e922f99d service cinder] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:13.419023 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6245df9-41f0-4cdc-9887-33c0e922f99d service cinder] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:13.419540 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6245df9-41f0-4cdc-9887-33c0e922f99d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:13.502733 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6245df9-41f0-4cdc-9887-33c0e922f99d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:19:13.504243 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6245df9-41f0-4cdc-9887-33c0e922f99d service cinder] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:19:13.516982 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1038/2098] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:19:13 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:19:15.283420 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5674793-4fd2-4b3f-8c16-50f776965ac7 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:19:15.284818 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-f5674793-4fd2-4b3f-8c16-50f776965ac7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:19:15.297401 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5674793-4fd2-4b3f-8c16-50f776965ac7 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:19:15.301290 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6073ffab-eff7-4f69-9630-304f7f92f3ef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e76f1952a5b146808844b9f2b7395035', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:19:15.301953 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6073ffab-eff7-4f69-9630-304f7f92f3ef service cinder] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:15.302255 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6073ffab-eff7-4f69-9630-304f7f92f3ef service cinder] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:15.302536 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6073ffab-eff7-4f69-9630-304f7f92f3ef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:15.360956 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:19:15.363721 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6073ffab-eff7-4f69-9630-304f7f92f3ef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:19:15.366304 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6073ffab-eff7-4f69-9630-304f7f92f3ef service cinder] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:19:15.376817 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1061/2099] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:19:15 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:19:15.958568 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6245df9-41f0-4cdc-9887-33c0e922f99d service cinder] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:19:15.967461 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6245df9-41f0-4cdc-9887-33c0e922f99d service cinder] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:19:15.972009 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5359103a-4720-41ea-87fd-dec1ff01a027 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e76f1952a5b146808844b9f2b7395035', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:19:15.972651 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5359103a-4720-41ea-87fd-dec1ff01a027 service cinder] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:15.972930 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5359103a-4720-41ea-87fd-dec1ff01a027 service cinder] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:15.973200 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5359103a-4720-41ea-87fd-dec1ff01a027 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:15.982914 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5359103a-4720-41ea-87fd-dec1ff01a027 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:19:15.984458 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5359103a-4720-41ea-87fd-dec1ff01a027 service cinder] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:19:15.995606 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1039/2100] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:19:15 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:19:16.296262 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6073ffab-eff7-4f69-9630-304f7f92f3ef service cinder] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:19:16.297968 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-6073ffab-eff7-4f69-9630-304f7f92f3ef service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:19:16.386611 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6073ffab-eff7-4f69-9630-304f7f92f3ef service cinder] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:19:16.391832 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9689d6f6-14e8-4039-8dd2-42462738c4b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:19:16.392702 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9689d6f6-14e8-4039-8dd2-42462738c4b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:16.392984 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9689d6f6-14e8-4039-8dd2-42462738c4b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:16.393254 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9689d6f6-14e8-4039-8dd2-42462738c4b9 admin admin] PATH_INFO: `/v3/users/5b7f99f7e7aa48df9e84f5ce5506b867` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:16.394819 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9689d6f6-14e8-4039-8dd2-42462738c4b9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5b7f99f7e7aa48df9e84f5ce5506b867)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:19:16.397272 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9689d6f6-14e8-4039-8dd2-42462738c4b9 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:19:16.518106 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-9689d6f6-14e8-4039-8dd2-42462738c4b9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5b7f99f7e7aa48df9e84f5ce5506b867'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:19:16.519244 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-9689d6f6-14e8-4039-8dd2-42462738c4b9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5b7f99f7e7aa48df9e84f5ce5506b867'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:19:16.544327 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-9689d6f6-14e8-4039-8dd2-42462738c4b9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5b7f99f7e7aa48df9e84f5ce5506b867'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:19:16.559671 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-9689d6f6-14e8-4039-8dd2-42462738c4b9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5b7f99f7e7aa48df9e84f5ce5506b867'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:19:16.560603 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-9689d6f6-14e8-4039-8dd2-42462738c4b9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5b7f99f7e7aa48df9e84f5ce5506b867'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:19:16.580200 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1062/2101] 10.222.0.26 () {58 vars in 1223 bytes} [Thu Jul 14 09:19:16 2022] DELETE /identity/v3/users/5b7f99f7e7aa48df9e84f5ce5506b867 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:19:16.584650 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e94bb7-2f3c-432c-a4d0-f530ac562db1 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:16.584969 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e94bb7-2f3c-432c-a4d0-f530ac562db1 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:16.585281 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e94bb7-2f3c-432c-a4d0-f530ac562db1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:16.604267 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-47e94bb7-2f3c-432c-a4d0-f530ac562db1 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:19:16.606355 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-47e94bb7-2f3c-432c-a4d0-f530ac562db1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:19:16.712873 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1040/2102] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:19:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:19:16.723619 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9689d6f6-14e8-4039-8dd2-42462738c4b9 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:19:16.728300 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-9689d6f6-14e8-4039-8dd2-42462738c4b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:19:16.847528 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9689d6f6-14e8-4039-8dd2-42462738c4b9 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:19:16.851917 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a044a577-fd35-4a39-9a04-d78137e6a1be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '952911c492444ffea4b211a24b564117', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:19:16.855933 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a044a577-fd35-4a39-9a04-d78137e6a1be service neutron] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:16.856247 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a044a577-fd35-4a39-9a04-d78137e6a1be service neutron] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:16.856533 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a044a577-fd35-4a39-9a04-d78137e6a1be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:16.880113 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a044a577-fd35-4a39-9a04-d78137e6a1be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:19:16.882329 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a044a577-fd35-4a39-9a04-d78137e6a1be service neutron] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:19:16.902545 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1063/2103] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:19:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:19:17.438541 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e94bb7-2f3c-432c-a4d0-f530ac562db1 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:19:17.440368 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-47e94bb7-2f3c-432c-a4d0-f530ac562db1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:19:17.458312 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e94bb7-2f3c-432c-a4d0-f530ac562db1 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:19:17.464599 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84fb653-ec90-491e-b352-161d102aa7f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:19:17.465653 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84fb653-ec90-491e-b352-161d102aa7f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:17.466080 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84fb653-ec90-491e-b352-161d102aa7f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:17.466496 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84fb653-ec90-491e-b352-161d102aa7f3 admin admin] PATH_INFO: `/v3/projects/2d724787c4b3469db2a80ba744b51f7c` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:17.468449 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84fb653-ec90-491e-b352-161d102aa7f3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2d724787c4b3469db2a80ba744b51f7c)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:19:17.471570 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84fb653-ec90-491e-b352-161d102aa7f3 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:19:17.529566 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-a84fb653-ec90-491e-b352-161d102aa7f3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2d724787c4b3469db2a80ba744b51f7c', 'request_id': 'req-a84fb653-ec90-491e-b352-161d102aa7f3'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:19:17.541417 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-a84fb653-ec90-491e-b352-161d102aa7f3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2d724787c4b3469db2a80ba744b51f7c', 'request_id': 'req-a84fb653-ec90-491e-b352-161d102aa7f3'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:19:17.547909 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-a84fb653-ec90-491e-b352-161d102aa7f3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2d724787c4b3469db2a80ba744b51f7c', 'request_id': 'req-a84fb653-ec90-491e-b352-161d102aa7f3'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:19:17.560058 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:19:17.565939 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-a84fb653-ec90-491e-b352-161d102aa7f3 admin admin] The token cache is being invalidate because project 2d724787c4b3469db2a80ba744b51f7c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80381) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 09:19:17.569788 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-a84fb653-ec90-491e-b352-161d102aa7f3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:19:17.570384 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-a84fb653-ec90-491e-b352-161d102aa7f3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:19:17.574995 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1041/2104] 10.222.0.26 () {58 vars in 1232 bytes} [Thu Jul 14 09:19:17 2022] DELETE /identity/v3/projects/2d724787c4b3469db2a80ba744b51f7c => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:19:17.576361 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a044a577-fd35-4a39-9a04-d78137e6a1be service neutron] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:19:17.590738 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a044a577-fd35-4a39-9a04-d78137e6a1be service neutron] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:19:17.617992 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:19:17.667136 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a106ca-3b0f-44b6-800b-72dd0dd61326 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:19:17.668064 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a106ca-3b0f-44b6-800b-72dd0dd61326 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:17.668345 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a106ca-3b0f-44b6-800b-72dd0dd61326 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:17.668620 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a106ca-3b0f-44b6-800b-72dd0dd61326 admin admin] PATH_INFO: `/v3/users/0a42f772e8e84e6b8f7e5ce67bb9009d` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:17.669939 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a106ca-3b0f-44b6-800b-72dd0dd61326 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0a42f772e8e84e6b8f7e5ce67bb9009d)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:19:17.671956 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a106ca-3b0f-44b6-800b-72dd0dd61326 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:19:17.721144 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-36a106ca-3b0f-44b6-800b-72dd0dd61326 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0a42f772e8e84e6b8f7e5ce67bb9009d'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:19:17.721984 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-36a106ca-3b0f-44b6-800b-72dd0dd61326 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0a42f772e8e84e6b8f7e5ce67bb9009d'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:19:17.734566 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-36a106ca-3b0f-44b6-800b-72dd0dd61326 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0a42f772e8e84e6b8f7e5ce67bb9009d'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:19:17.742868 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-36a106ca-3b0f-44b6-800b-72dd0dd61326 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0a42f772e8e84e6b8f7e5ce67bb9009d'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:19:17.743529 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-36a106ca-3b0f-44b6-800b-72dd0dd61326 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0a42f772e8e84e6b8f7e5ce67bb9009d'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:19:17.755726 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1064/2105] 10.222.0.26 () {58 vars in 1223 bytes} [Thu Jul 14 09:19:17 2022] DELETE /identity/v3/users/0a42f772e8e84e6b8f7e5ce67bb9009d => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:19:18.045885 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84fb653-ec90-491e-b352-161d102aa7f3 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:19:18.051825 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-a84fb653-ec90-491e-b352-161d102aa7f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:19:18.086383 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84fb653-ec90-491e-b352-161d102aa7f3 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:19:18.146536 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0c4931-6b81-4518-80fb-ae6b3f1c37f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:19:18.147740 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0c4931-6b81-4518-80fb-ae6b3f1c37f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:18.148056 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0c4931-6b81-4518-80fb-ae6b3f1c37f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:18.148324 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0c4931-6b81-4518-80fb-ae6b3f1c37f3 admin admin] PATH_INFO: `/v3/projects/7584e47cf97e46768eacf7900991bb2d` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:18.149938 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0c4931-6b81-4518-80fb-ae6b3f1c37f3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7584e47cf97e46768eacf7900991bb2d)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:19:18.151900 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0c4931-6b81-4518-80fb-ae6b3f1c37f3 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:19:18.205767 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-df0c4931-6b81-4518-80fb-ae6b3f1c37f3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7584e47cf97e46768eacf7900991bb2d', 'request_id': 'req-df0c4931-6b81-4518-80fb-ae6b3f1c37f3'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:19:18.216964 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-df0c4931-6b81-4518-80fb-ae6b3f1c37f3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7584e47cf97e46768eacf7900991bb2d', 'request_id': 'req-df0c4931-6b81-4518-80fb-ae6b3f1c37f3'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:19:18.223795 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-df0c4931-6b81-4518-80fb-ae6b3f1c37f3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7584e47cf97e46768eacf7900991bb2d', 'request_id': 'req-df0c4931-6b81-4518-80fb-ae6b3f1c37f3'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:19:18.244081 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-df0c4931-6b81-4518-80fb-ae6b3f1c37f3 admin admin] The token cache is being invalidate because project 7584e47cf97e46768eacf7900991bb2d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80381) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 09:19:18.244372 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-df0c4931-6b81-4518-80fb-ae6b3f1c37f3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:19:18.245018 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-df0c4931-6b81-4518-80fb-ae6b3f1c37f3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:19:18.246786 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1042/2106] 10.222.0.26 () {58 vars in 1232 bytes} [Thu Jul 14 09:19:18 2022] DELETE /identity/v3/projects/7584e47cf97e46768eacf7900991bb2d => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:19:18.252488 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2f8b2c-1c92-47e0-bba7-2f2b4b143a50 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:18.252777 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2f8b2c-1c92-47e0-bba7-2f2b4b143a50 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:18.253056 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2f8b2c-1c92-47e0-bba7-2f2b4b143a50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:18.271870 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-9c2f8b2c-1c92-47e0-bba7-2f2b4b143a50 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:19:18.273390 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-9c2f8b2c-1c92-47e0-bba7-2f2b4b143a50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:19:18.346630 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1065/2107] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:19:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:19:18.349647 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3354b99-288c-4d34-aea1-8012e40b735f None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:18.349967 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3354b99-288c-4d34-aea1-8012e40b735f None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:18.350258 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3354b99-288c-4d34-aea1-8012e40b735f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:18.369896 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-a3354b99-288c-4d34-aea1-8012e40b735f None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:19:18.371117 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-a3354b99-288c-4d34-aea1-8012e40b735f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:19:18.379074 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1043/2108] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:19:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:19:18.383327 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2f8b2c-1c92-47e0-bba7-2f2b4b143a50 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:19:18.393258 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2f8b2c-1c92-47e0-bba7-2f2b4b143a50 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:19:18.398896 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f06aaf-714d-427a-99c4-f62cf6404100 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:19:18.399782 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f06aaf-714d-427a-99c4-f62cf6404100 admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:18.400176 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f06aaf-714d-427a-99c4-f62cf6404100 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:18.400432 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f06aaf-714d-427a-99c4-f62cf6404100 admin admin] PATH_INFO: `/v3/domains` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:18.401319 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f06aaf-714d-427a-99c4-f62cf6404100 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:19:18.401631 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f06aaf-714d-427a-99c4-f62cf6404100 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:19:18.403497 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f06aaf-714d-427a-99c4-f62cf6404100 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:19:18.419295 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1066/2109] 10.222.0.26 () {58 vars in 1152 bytes} [Thu Jul 14 09:19:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:19:18.421231 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3354b99-288c-4d34-aea1-8012e40b735f None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:19:18.435032 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3354b99-288c-4d34-aea1-8012e40b735f None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:19:18.444308 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b38724-81c5-4b4a-acd9-713386d985e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:19:18.444987 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b38724-81c5-4b4a-acd9-713386d985e3 admin admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:18.445272 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b38724-81c5-4b4a-acd9-713386d985e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:18.445549 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b38724-81c5-4b4a-acd9-713386d985e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:18.446264 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b38724-81c5-4b4a-acd9-713386d985e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:19:18.448570 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b38724-81c5-4b4a-acd9-713386d985e3 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:19:18.482891 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1044/2110] 10.222.0.26 () {60 vars in 1152 bytes} [Thu Jul 14 09:19:18 2022] POST /identity/v3/projects => generated 375 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 09:19:18.484045 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f06aaf-714d-427a-99c4-f62cf6404100 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:19:18.494148 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f06aaf-714d-427a-99c4-f62cf6404100 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:19:18.515499 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:19:18.560475 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f441d75-22ad-4f5b-b487-58c7f283fc15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:19:18.561497 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f441d75-22ad-4f5b-b487-58c7f283fc15 admin admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:18.561974 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f441d75-22ad-4f5b-b487-58c7f283fc15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:18.562462 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f441d75-22ad-4f5b-b487-58c7f283fc15 admin admin] PATH_INFO: `/v3/users` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:18.564085 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f441d75-22ad-4f5b-b487-58c7f283fc15 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:19:18.566700 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f441d75-22ad-4f5b-b487-58c7f283fc15 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:19:18.589249 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1067/2111] 10.222.0.26 () {60 vars in 1143 bytes} [Thu Jul 14 09:19:18 2022] POST /identity/v3/users => generated 288 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 09:19:18.594852 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b38724-81c5-4b4a-acd9-713386d985e3 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:19:18.608359 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b38724-81c5-4b4a-acd9-713386d985e3 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:19:18.615378 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183f6722-7454-4dd6-9882-66bb48266fdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:19:18.615997 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183f6722-7454-4dd6-9882-66bb48266fdc admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:18.616454 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183f6722-7454-4dd6-9882-66bb48266fdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:18.616738 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183f6722-7454-4dd6-9882-66bb48266fdc admin admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:18.617744 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183f6722-7454-4dd6-9882-66bb48266fdc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:19:18.619453 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183f6722-7454-4dd6-9882-66bb48266fdc admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:19:18.633948 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1045/2112] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jul 14 09:19:18 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 09:19:18.635145 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f441d75-22ad-4f5b-b487-58c7f283fc15 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:19:18.645999 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f441d75-22ad-4f5b-b487-58c7f283fc15 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:19:18.651613 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb18fbe-a182-42a7-b528-376de69710fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:19:18.652401 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb18fbe-a182-42a7-b528-376de69710fc admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:18.652849 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb18fbe-a182-42a7-b528-376de69710fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:18.653127 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb18fbe-a182-42a7-b528-376de69710fc admin admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:18.653865 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb18fbe-a182-42a7-b528-376de69710fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:19:18.655755 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb18fbe-a182-42a7-b528-376de69710fc admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:19:18.667232 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1068/2113] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jul 14 09:19:18 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 09:19:18.671001 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183f6722-7454-4dd6-9882-66bb48266fdc admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:19:18.681286 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183f6722-7454-4dd6-9882-66bb48266fdc admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:19:18.686131 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4597fb-2839-46c8-b252-de7c5de9ddbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:19:18.687022 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4597fb-2839-46c8-b252-de7c5de9ddbf admin admin] REQUEST_METHOD: `PUT` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:18.687344 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4597fb-2839-46c8-b252-de7c5de9ddbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:18.687623 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4597fb-2839-46c8-b252-de7c5de9ddbf admin admin] PATH_INFO: `/v3/projects/03d5985494c34df19df17a31fe357d03/users/4389280de43243e08d5c2a297b6e4a09/roles/d1691322ce6a4edab7ce722edf0f2988` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:18.704559 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4597fb-2839-46c8-b252-de7c5de9ddbf admin admin] RBAC: Authorizing `identity:create_grant(project_id=03d5985494c34df19df17a31fe357d03, user_id=4389280de43243e08d5c2a297b6e4a09, role_id=d1691322ce6a4edab7ce722edf0f2988)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:19:18.707211 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4597fb-2839-46c8-b252-de7c5de9ddbf admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:19:18.720749 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1046/2114] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jul 14 09:19:18 2022] PUT /identity/v3/projects/03d5985494c34df19df17a31fe357d03/users/4389280de43243e08d5c2a297b6e4a09/roles/d1691322ce6a4edab7ce722edf0f2988 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 09:19:18.735017 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3660576f-cd83-42d0-976f-4a391100ff27 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:18.735331 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3660576f-cd83-42d0-976f-4a391100ff27 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:18.735974 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3660576f-cd83-42d0-976f-4a391100ff27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:18.762438 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-3660576f-cd83-42d0-976f-4a391100ff27 None None] MFA Rules not processed for user `4389280de43243e08d5c2a297b6e4a09`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:19:18.763704 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-3660576f-cd83-42d0-976f-4a391100ff27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:19:18.833793 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1069/2115] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:19:18 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:19:18.845018 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4597fb-2839-46c8-b252-de7c5de9ddbf admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:19:18.846721 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-3b4597fb-2839-46c8-b252-de7c5de9ddbf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:19:18.875071 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:19:18.911088 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4597fb-2839-46c8-b252-de7c5de9ddbf admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:19:18.915088 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aea11ee-758b-4fb9-9471-0f4e7e0b4c51 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e76f1952a5b146808844b9f2b7395035', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:19:18.915764 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aea11ee-758b-4fb9-9471-0f4e7e0b4c51 service cinder] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:18.916051 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aea11ee-758b-4fb9-9471-0f4e7e0b4c51 service cinder] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:18.916329 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aea11ee-758b-4fb9-9471-0f4e7e0b4c51 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:18.924962 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aea11ee-758b-4fb9-9471-0f4e7e0b4c51 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:19:18.926388 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aea11ee-758b-4fb9-9471-0f4e7e0b4c51 service cinder] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:19:18.939326 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1047/2116] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:19:18 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:19:22.033680 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3660576f-cd83-42d0-976f-4a391100ff27 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:19:22.035405 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-3660576f-cd83-42d0-976f-4a391100ff27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:19:22.103026 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3660576f-cd83-42d0-976f-4a391100ff27 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:19:22.108032 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb0797c-776e-47b5-bd11-5b665b6ad767 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:19:22.108937 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb0797c-776e-47b5-bd11-5b665b6ad767 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:22.109232 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb0797c-776e-47b5-bd11-5b665b6ad767 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:22.109501 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb0797c-776e-47b5-bd11-5b665b6ad767 admin admin] PATH_INFO: `/v3/users/de74ccad93f742e6b43e4c4cd4550986` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:22.110695 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb0797c-776e-47b5-bd11-5b665b6ad767 admin admin] RBAC: Authorizing `identity:delete_user(user_id=de74ccad93f742e6b43e4c4cd4550986)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:19:22.112692 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb0797c-776e-47b5-bd11-5b665b6ad767 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:19:22.156196 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:19:22.168060 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-7eb0797c-776e-47b5-bd11-5b665b6ad767 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de74ccad93f742e6b43e4c4cd4550986'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:19:22.168776 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-7eb0797c-776e-47b5-bd11-5b665b6ad767 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de74ccad93f742e6b43e4c4cd4550986'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:19:22.189338 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-7eb0797c-776e-47b5-bd11-5b665b6ad767 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de74ccad93f742e6b43e4c4cd4550986'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:19:22.207846 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-7eb0797c-776e-47b5-bd11-5b665b6ad767 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de74ccad93f742e6b43e4c4cd4550986'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:19:22.208680 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-7eb0797c-776e-47b5-bd11-5b665b6ad767 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de74ccad93f742e6b43e4c4cd4550986'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:19:22.223865 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1070/2117] 10.222.0.26 () {58 vars in 1223 bytes} [Thu Jul 14 09:19:22 2022] DELETE /identity/v3/users/de74ccad93f742e6b43e4c4cd4550986 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:19:22.228423 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33878e4-264a-4851-ba4c-dcf1b9233395 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:22.228884 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33878e4-264a-4851-ba4c-dcf1b9233395 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:22.228977 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33878e4-264a-4851-ba4c-dcf1b9233395 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:22.248210 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-d33878e4-264a-4851-ba4c-dcf1b9233395 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:19:22.249660 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-d33878e4-264a-4851-ba4c-dcf1b9233395 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:19:22.339066 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1048/2118] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:19:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:19:22.348729 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb0797c-776e-47b5-bd11-5b665b6ad767 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:19:22.350207 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-7eb0797c-776e-47b5-bd11-5b665b6ad767 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:19:22.444316 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb0797c-776e-47b5-bd11-5b665b6ad767 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:19:22.448036 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ceb0d3-e787-475e-9196-3bf7fdc5ae62 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '952911c492444ffea4b211a24b564117', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:19:22.448742 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ceb0d3-e787-475e-9196-3bf7fdc5ae62 service neutron] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:22.449024 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ceb0d3-e787-475e-9196-3bf7fdc5ae62 service neutron] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:22.449297 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ceb0d3-e787-475e-9196-3bf7fdc5ae62 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:22.459972 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ceb0d3-e787-475e-9196-3bf7fdc5ae62 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:19:22.461498 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ceb0d3-e787-475e-9196-3bf7fdc5ae62 service neutron] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:19:22.471590 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1071/2119] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:19:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:19:22.770714 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33878e4-264a-4851-ba4c-dcf1b9233395 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:19:22.772189 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-d33878e4-264a-4851-ba4c-dcf1b9233395 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:19:22.787525 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33878e4-264a-4851-ba4c-dcf1b9233395 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:19:22.799949 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea5991e-dd3c-49ee-9fe3-a2edb2a6dad1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:19:22.802980 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea5991e-dd3c-49ee-9fe3-a2edb2a6dad1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:22.803475 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea5991e-dd3c-49ee-9fe3-a2edb2a6dad1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:22.804000 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea5991e-dd3c-49ee-9fe3-a2edb2a6dad1 admin admin] PATH_INFO: `/v3/projects/349f91a94f264213a1e9fdcf847e93a4` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:22.805751 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea5991e-dd3c-49ee-9fe3-a2edb2a6dad1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=349f91a94f264213a1e9fdcf847e93a4)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:19:22.808017 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea5991e-dd3c-49ee-9fe3-a2edb2a6dad1 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:19:22.957200 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-eea5991e-dd3c-49ee-9fe3-a2edb2a6dad1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '349f91a94f264213a1e9fdcf847e93a4', 'request_id': 'req-eea5991e-dd3c-49ee-9fe3-a2edb2a6dad1'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:19:22.970040 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-eea5991e-dd3c-49ee-9fe3-a2edb2a6dad1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '349f91a94f264213a1e9fdcf847e93a4', 'request_id': 'req-eea5991e-dd3c-49ee-9fe3-a2edb2a6dad1'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:19:22.980560 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-eea5991e-dd3c-49ee-9fe3-a2edb2a6dad1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '349f91a94f264213a1e9fdcf847e93a4', 'request_id': 'req-eea5991e-dd3c-49ee-9fe3-a2edb2a6dad1'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:19:23.009673 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-eea5991e-dd3c-49ee-9fe3-a2edb2a6dad1 admin admin] The token cache is being invalidate because project 349f91a94f264213a1e9fdcf847e93a4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80381) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 09:19:23.010076 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-eea5991e-dd3c-49ee-9fe3-a2edb2a6dad1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:19:23.010815 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-eea5991e-dd3c-49ee-9fe3-a2edb2a6dad1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:19:23.013311 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1049/2120] 10.222.0.26 () {58 vars in 1232 bytes} [Thu Jul 14 09:19:22 2022] DELETE /identity/v3/projects/349f91a94f264213a1e9fdcf847e93a4 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:19:23.016577 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ceb0d3-e787-475e-9196-3bf7fdc5ae62 service neutron] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:19:23.030233 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ceb0d3-e787-475e-9196-3bf7fdc5ae62 service neutron] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:19:23.126703 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee843939-83da-4061-8ea0-e15a3a3d7e40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:19:23.127916 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee843939-83da-4061-8ea0-e15a3a3d7e40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:23.128268 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee843939-83da-4061-8ea0-e15a3a3d7e40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:23.128610 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee843939-83da-4061-8ea0-e15a3a3d7e40 admin admin] PATH_INFO: `/v3/users/7bd7ab75c8bc4f4caf0383f22246e2ed` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:23.129929 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee843939-83da-4061-8ea0-e15a3a3d7e40 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7bd7ab75c8bc4f4caf0383f22246e2ed)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:19:23.132371 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee843939-83da-4061-8ea0-e15a3a3d7e40 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:19:23.181596 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-ee843939-83da-4061-8ea0-e15a3a3d7e40 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7bd7ab75c8bc4f4caf0383f22246e2ed'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:19:23.182245 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-ee843939-83da-4061-8ea0-e15a3a3d7e40 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7bd7ab75c8bc4f4caf0383f22246e2ed'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:19:23.189686 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:19:23.196044 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-ee843939-83da-4061-8ea0-e15a3a3d7e40 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7bd7ab75c8bc4f4caf0383f22246e2ed'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:19:23.205787 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-ee843939-83da-4061-8ea0-e15a3a3d7e40 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7bd7ab75c8bc4f4caf0383f22246e2ed'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:19:23.206484 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-ee843939-83da-4061-8ea0-e15a3a3d7e40 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7bd7ab75c8bc4f4caf0383f22246e2ed'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:19:23.220079 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1072/2121] 10.222.0.26 () {58 vars in 1223 bytes} [Thu Jul 14 09:19:23 2022] DELETE /identity/v3/users/7bd7ab75c8bc4f4caf0383f22246e2ed => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:19:23.553000 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea5991e-dd3c-49ee-9fe3-a2edb2a6dad1 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:19:23.554572 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-eea5991e-dd3c-49ee-9fe3-a2edb2a6dad1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:19:23.580490 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea5991e-dd3c-49ee-9fe3-a2edb2a6dad1 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:19:23.650106 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:19:23.671757 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808d6080-9bf0-4946-8780-31d6682e7841 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:19:23.672735 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808d6080-9bf0-4946-8780-31d6682e7841 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:23.672903 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808d6080-9bf0-4946-8780-31d6682e7841 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:23.674104 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808d6080-9bf0-4946-8780-31d6682e7841 admin admin] PATH_INFO: `/v3/projects/3a4bd179449c4c4d85386f4ba069c775` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:23.675631 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808d6080-9bf0-4946-8780-31d6682e7841 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3a4bd179449c4c4d85386f4ba069c775)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:19:23.678317 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808d6080-9bf0-4946-8780-31d6682e7841 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:19:23.725601 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-808d6080-9bf0-4946-8780-31d6682e7841 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3a4bd179449c4c4d85386f4ba069c775', 'request_id': 'req-808d6080-9bf0-4946-8780-31d6682e7841'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:19:23.731709 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-808d6080-9bf0-4946-8780-31d6682e7841 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3a4bd179449c4c4d85386f4ba069c775', 'request_id': 'req-808d6080-9bf0-4946-8780-31d6682e7841'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:19:23.737126 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-808d6080-9bf0-4946-8780-31d6682e7841 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3a4bd179449c4c4d85386f4ba069c775', 'request_id': 'req-808d6080-9bf0-4946-8780-31d6682e7841'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:19:23.751017 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-808d6080-9bf0-4946-8780-31d6682e7841 admin admin] The token cache is being invalidate because project 3a4bd179449c4c4d85386f4ba069c775 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80381) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 09:19:23.752238 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-808d6080-9bf0-4946-8780-31d6682e7841 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:19:23.752853 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-808d6080-9bf0-4946-8780-31d6682e7841 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:19:23.754390 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1050/2122] 10.222.0.26 () {58 vars in 1232 bytes} [Thu Jul 14 09:19:23 2022] DELETE /identity/v3/projects/3a4bd179449c4c4d85386f4ba069c775 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:19:23.758427 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee843939-83da-4061-8ea0-e15a3a3d7e40 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:19:23.759936 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-ee843939-83da-4061-8ea0-e15a3a3d7e40 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:19:23.773486 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee843939-83da-4061-8ea0-e15a3a3d7e40 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:19:23.839833 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32def4b9-8105-4199-807b-a88939912142 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:19:23.840775 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32def4b9-8105-4199-807b-a88939912142 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:23.841058 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32def4b9-8105-4199-807b-a88939912142 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:23.841334 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32def4b9-8105-4199-807b-a88939912142 admin admin] PATH_INFO: `/v3/users/0a3d2ce3e6d14de184e51d13043aa4e0` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:23.842721 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32def4b9-8105-4199-807b-a88939912142 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0a3d2ce3e6d14de184e51d13043aa4e0)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:19:23.844736 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32def4b9-8105-4199-807b-a88939912142 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:19:23.894709 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-32def4b9-8105-4199-807b-a88939912142 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0a3d2ce3e6d14de184e51d13043aa4e0'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:19:23.895398 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-32def4b9-8105-4199-807b-a88939912142 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0a3d2ce3e6d14de184e51d13043aa4e0'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:19:23.908585 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-32def4b9-8105-4199-807b-a88939912142 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0a3d2ce3e6d14de184e51d13043aa4e0'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:19:23.916961 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-32def4b9-8105-4199-807b-a88939912142 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0a3d2ce3e6d14de184e51d13043aa4e0'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:19:23.917626 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-32def4b9-8105-4199-807b-a88939912142 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0a3d2ce3e6d14de184e51d13043aa4e0'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:19:23.930861 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1073/2123] 10.222.0.26 () {58 vars in 1223 bytes} [Thu Jul 14 09:19:23 2022] DELETE /identity/v3/users/0a3d2ce3e6d14de184e51d13043aa4e0 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:19:24.263874 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808d6080-9bf0-4946-8780-31d6682e7841 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:19:24.270036 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-808d6080-9bf0-4946-8780-31d6682e7841 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:19:24.293190 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808d6080-9bf0-4946-8780-31d6682e7841 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:19:24.328838 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32def4b9-8105-4199-807b-a88939912142 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:19:24.330870 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-32def4b9-8105-4199-807b-a88939912142 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:19:24.375747 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1915be24-c6b9-4a42-ae9d-93f1c274678c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:19:24.376631 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1915be24-c6b9-4a42-ae9d-93f1c274678c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:24.376917 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1915be24-c6b9-4a42-ae9d-93f1c274678c admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:24.377192 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1915be24-c6b9-4a42-ae9d-93f1c274678c admin admin] PATH_INFO: `/v3/users/4389280de43243e08d5c2a297b6e4a09` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:24.378426 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1915be24-c6b9-4a42-ae9d-93f1c274678c admin admin] RBAC: Authorizing `identity:delete_user(user_id=4389280de43243e08d5c2a297b6e4a09)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:19:24.380412 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1915be24-c6b9-4a42-ae9d-93f1c274678c admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:19:24.415364 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:19:24.435084 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32def4b9-8105-4199-807b-a88939912142 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:19:24.440413 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3964b052-9183-47bd-b150-7c92322abbcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:19:24.441416 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3964b052-9183-47bd-b150-7c92322abbcb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:24.441917 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3964b052-9183-47bd-b150-7c92322abbcb admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:24.442200 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3964b052-9183-47bd-b150-7c92322abbcb admin admin] PATH_INFO: `/v3/projects/a2e13ff34a3549f4b34509833e5f073e` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:24.443768 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3964b052-9183-47bd-b150-7c92322abbcb admin admin] RBAC: Authorizing `identity:delete_project(project_id=a2e13ff34a3549f4b34509833e5f073e)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:19:24.445271 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-1915be24-c6b9-4a42-ae9d-93f1c274678c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4389280de43243e08d5c2a297b6e4a09'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:19:24.445914 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3964b052-9183-47bd-b150-7c92322abbcb admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:19:24.455939 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-1915be24-c6b9-4a42-ae9d-93f1c274678c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4389280de43243e08d5c2a297b6e4a09'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:19:24.457096 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-1915be24-c6b9-4a42-ae9d-93f1c274678c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4389280de43243e08d5c2a297b6e4a09'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:19:24.458232 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-1915be24-c6b9-4a42-ae9d-93f1c274678c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4389280de43243e08d5c2a297b6e4a09'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:19:24.466644 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-1915be24-c6b9-4a42-ae9d-93f1c274678c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4389280de43243e08d5c2a297b6e4a09'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:19:24.482980 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:19:24.487234 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1051/2124] 10.222.0.26 () {58 vars in 1223 bytes} [Thu Jul 14 09:19:24 2022] DELETE /identity/v3/users/4389280de43243e08d5c2a297b6e4a09 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:19:24.491587 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e794f1-a1ec-411d-b032-680d5c626910 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:24.491831 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e794f1-a1ec-411d-b032-680d5c626910 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:24.492837 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e794f1-a1ec-411d-b032-680d5c626910 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:24.501064 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-3964b052-9183-47bd-b150-7c92322abbcb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a2e13ff34a3549f4b34509833e5f073e', 'request_id': 'req-3964b052-9183-47bd-b150-7c92322abbcb'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:19:24.507573 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-3964b052-9183-47bd-b150-7c92322abbcb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a2e13ff34a3549f4b34509833e5f073e', 'request_id': 'req-3964b052-9183-47bd-b150-7c92322abbcb'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:19:24.511693 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-73e794f1-a1ec-411d-b032-680d5c626910 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:19:24.512887 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-73e794f1-a1ec-411d-b032-680d5c626910 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:19:24.513295 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-3964b052-9183-47bd-b150-7c92322abbcb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a2e13ff34a3549f4b34509833e5f073e', 'request_id': 'req-3964b052-9183-47bd-b150-7c92322abbcb'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:19:24.526543 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-3964b052-9183-47bd-b150-7c92322abbcb admin admin] The token cache is being invalidate because project a2e13ff34a3549f4b34509833e5f073e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80382) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 09:19:24.526833 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-3964b052-9183-47bd-b150-7c92322abbcb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:19:24.527483 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-3964b052-9183-47bd-b150-7c92322abbcb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:19:24.529287 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1074/2125] 10.222.0.26 () {58 vars in 1232 bytes} [Thu Jul 14 09:19:24 2022] DELETE /identity/v3/projects/a2e13ff34a3549f4b34509833e5f073e => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:19:24.533927 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e16b167-55f2-4e93-bf8e-c3a27e50d556 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:24.534367 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e16b167-55f2-4e93-bf8e-c3a27e50d556 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:24.534656 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e16b167-55f2-4e93-bf8e-c3a27e50d556 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:24.552374 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-0e16b167-55f2-4e93-bf8e-c3a27e50d556 None None] MFA Rules not processed for user `7d6267ede8d64e2abe1f4058a9797fb7`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:19:24.553649 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-0e16b167-55f2-4e93-bf8e-c3a27e50d556 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:19:24.581794 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1052/2126] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:19:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:19:24.584140 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2646fbb0-f0a6-47ea-a681-ce69bebeec96 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:24.584432 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2646fbb0-f0a6-47ea-a681-ce69bebeec96 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:24.584790 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2646fbb0-f0a6-47ea-a681-ce69bebeec96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:24.602348 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-2646fbb0-f0a6-47ea-a681-ce69bebeec96 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:19:24.603562 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-2646fbb0-f0a6-47ea-a681-ce69bebeec96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:19:24.610905 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1075/2127] 10.222.0.26 () {58 vars in 1009 bytes} [Thu Jul 14 09:19:24 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:19:24.611534 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1053/2128] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:19:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:19:24.611788 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e16b167-55f2-4e93-bf8e-c3a27e50d556 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:19:24.613071 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-0e16b167-55f2-4e93-bf8e-c3a27e50d556 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:19:24.619522 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6939c66-88be-4d25-9f8f-6be28bbf6f0b None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:24.619522 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6939c66-88be-4d25-9f8f-6be28bbf6f0b None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:24.619522 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6939c66-88be-4d25-9f8f-6be28bbf6f0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:24.642011 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-f6939c66-88be-4d25-9f8f-6be28bbf6f0b None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:19:24.643298 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-f6939c66-88be-4d25-9f8f-6be28bbf6f0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:19:24.652158 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1054/2129] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:19:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 09:19:24.656421 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6939c66-88be-4d25-9f8f-6be28bbf6f0b None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:19:24.666378 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6939c66-88be-4d25-9f8f-6be28bbf6f0b None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:19:24.671317 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabdc5c1-3ade-46e7-8e01-607986494001 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:19:24.671975 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabdc5c1-3ade-46e7-8e01-607986494001 admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:24.672313 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabdc5c1-3ade-46e7-8e01-607986494001 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:24.672592 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabdc5c1-3ade-46e7-8e01-607986494001 admin admin] PATH_INFO: `/v3/domains` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:24.673322 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabdc5c1-3ade-46e7-8e01-607986494001 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:19:24.673620 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabdc5c1-3ade-46e7-8e01-607986494001 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:19:24.675293 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabdc5c1-3ade-46e7-8e01-607986494001 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:19:24.675977 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e16b167-55f2-4e93-bf8e-c3a27e50d556 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:19:24.685514 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475c3698-5181-4543-9b13-5189a686a50f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '952911c492444ffea4b211a24b564117', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:19:24.686431 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475c3698-5181-4543-9b13-5189a686a50f service neutron] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:24.686720 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475c3698-5181-4543-9b13-5189a686a50f service neutron] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:24.686999 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475c3698-5181-4543-9b13-5189a686a50f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:24.687724 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1055/2130] 10.222.0.26 () {58 vars in 1152 bytes} [Thu Jul 14 09:19:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 09:19:24.688531 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabdc5c1-3ade-46e7-8e01-607986494001 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:19:24.690134 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-aabdc5c1-3ade-46e7-8e01-607986494001 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:19:24.708339 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475c3698-5181-4543-9b13-5189a686a50f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:19:24.710680 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475c3698-5181-4543-9b13-5189a686a50f service neutron] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:19:24.726127 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1076/2131] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:19:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:19:24.727241 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475c3698-5181-4543-9b13-5189a686a50f service neutron] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:19:24.738854 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475c3698-5181-4543-9b13-5189a686a50f service neutron] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:19:24.747389 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925b6f66-be66-4a16-9c4d-3d44f57f55db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:19:24.747656 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925b6f66-be66-4a16-9c4d-3d44f57f55db admin admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:24.748052 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925b6f66-be66-4a16-9c4d-3d44f57f55db admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:24.748519 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925b6f66-be66-4a16-9c4d-3d44f57f55db admin admin] PATH_INFO: `/v3/projects` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:24.749098 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925b6f66-be66-4a16-9c4d-3d44f57f55db admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:19:24.752043 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925b6f66-be66-4a16-9c4d-3d44f57f55db admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:19:24.794757 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1077/2132] 10.222.0.26 () {60 vars in 1152 bytes} [Thu Jul 14 09:19:24 2022] POST /identity/v3/projects => generated 383 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 09:19:24.798856 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925b6f66-be66-4a16-9c4d-3d44f57f55db admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:19:24.810050 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabdc5c1-3ade-46e7-8e01-607986494001 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:19:24.821085 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:19:24.909070 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13e28db-8a69-44db-a5db-e59263ed8331 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5a43c80514b4a13af03c7f8ebcf7a5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:19:24.909961 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13e28db-8a69-44db-a5db-e59263ed8331 service swift] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:24.910143 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13e28db-8a69-44db-a5db-e59263ed8331 service swift] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:24.910468 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13e28db-8a69-44db-a5db-e59263ed8331 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:24.917139 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925b6f66-be66-4a16-9c4d-3d44f57f55db admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:19:24.922506 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03605812-03ea-404f-9c1d-e2c3915416e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:19:24.923428 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03605812-03ea-404f-9c1d-e2c3915416e9 admin admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:24.923641 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03605812-03ea-404f-9c1d-e2c3915416e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:24.923979 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03605812-03ea-404f-9c1d-e2c3915416e9 admin admin] PATH_INFO: `/v3/users` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:24.925200 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03605812-03ea-404f-9c1d-e2c3915416e9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:19:24.927268 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03605812-03ea-404f-9c1d-e2c3915416e9 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:19:24.931764 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:19:24.961876 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1078/2133] 10.222.0.26 () {60 vars in 1143 bytes} [Thu Jul 14 09:19:24 2022] POST /identity/v3/users => generated 292 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 09:19:24.962852 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03605812-03ea-404f-9c1d-e2c3915416e9 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:19:24.973186 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03605812-03ea-404f-9c1d-e2c3915416e9 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:19:24.978411 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6b3629-0cd8-4a52-8e7d-c18d309e4bc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:19:24.979222 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6b3629-0cd8-4a52-8e7d-c18d309e4bc2 admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:24.979581 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6b3629-0cd8-4a52-8e7d-c18d309e4bc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:24.979925 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6b3629-0cd8-4a52-8e7d-c18d309e4bc2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:24.980723 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6b3629-0cd8-4a52-8e7d-c18d309e4bc2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:19:24.982406 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6b3629-0cd8-4a52-8e7d-c18d309e4bc2 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:19:24.982694 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13e28db-8a69-44db-a5db-e59263ed8331 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:19:24.984311 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13e28db-8a69-44db-a5db-e59263ed8331 service swift] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:19:24.993767 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1056/2134] 10.222.0.26 () {60 vars in 1384 bytes} [Thu Jul 14 09:19:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 14 09:19:24.993919 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1079/2135] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jul 14 09:19:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 09:19:24.997680 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13e28db-8a69-44db-a5db-e59263ed8331 service swift] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:19:25.009000 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13e28db-8a69-44db-a5db-e59263ed8331 service swift] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:19:25.014198 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f7e7fa-c342-4a1a-8909-00271b7845c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:19:25.014937 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f7e7fa-c342-4a1a-8909-00271b7845c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:25.015253 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f7e7fa-c342-4a1a-8909-00271b7845c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:25.015916 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f7e7fa-c342-4a1a-8909-00271b7845c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:25.016271 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f7e7fa-c342-4a1a-8909-00271b7845c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:19:25.017880 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f7e7fa-c342-4a1a-8909-00271b7845c4 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:19:25.030787 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1057/2136] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jul 14 09:19:24 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 09:19:25.034875 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6b3629-0cd8-4a52-8e7d-c18d309e4bc2 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:19:25.048472 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6b3629-0cd8-4a52-8e7d-c18d309e4bc2 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:19:25.057211 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1704bf9a-4da3-425b-9363-b79245a5a86b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:19:25.058308 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1704bf9a-4da3-425b-9363-b79245a5a86b admin admin] REQUEST_METHOD: `PUT` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:25.058707 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1704bf9a-4da3-425b-9363-b79245a5a86b admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:25.059064 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1704bf9a-4da3-425b-9363-b79245a5a86b admin admin] PATH_INFO: `/v3/projects/2392e639b4814c6a9783bad07d2b1afb/users/d376a81b81cc45a6b9b1b5f4000e6d75/roles/d1691322ce6a4edab7ce722edf0f2988` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:25.071150 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f7e7fa-c342-4a1a-8909-00271b7845c4 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:19:25.073169 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-01f7e7fa-c342-4a1a-8909-00271b7845c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:19:25.078014 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1704bf9a-4da3-425b-9363-b79245a5a86b admin admin] RBAC: Authorizing `identity:create_grant(project_id=2392e639b4814c6a9783bad07d2b1afb, user_id=d376a81b81cc45a6b9b1b5f4000e6d75, role_id=d1691322ce6a4edab7ce722edf0f2988)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:19:25.086772 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1704bf9a-4da3-425b-9363-b79245a5a86b admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:19:25.092368 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f7e7fa-c342-4a1a-8909-00271b7845c4 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:19:25.098435 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77180801-259b-46a7-b1ed-72596aa82837 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:19:25.100106 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77180801-259b-46a7-b1ed-72596aa82837 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:25.100402 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77180801-259b-46a7-b1ed-72596aa82837 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:25.100757 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77180801-259b-46a7-b1ed-72596aa82837 admin admin] PATH_INFO: `/v3/projects/03d5985494c34df19df17a31fe357d03` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:25.102285 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77180801-259b-46a7-b1ed-72596aa82837 admin admin] RBAC: Authorizing `identity:delete_project(project_id=03d5985494c34df19df17a31fe357d03)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:19:25.104932 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77180801-259b-46a7-b1ed-72596aa82837 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:19:25.108859 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1080/2137] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jul 14 09:19:25 2022] PUT /identity/v3/projects/2392e639b4814c6a9783bad07d2b1afb/users/d376a81b81cc45a6b9b1b5f4000e6d75/roles/d1691322ce6a4edab7ce722edf0f2988 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:19:25.122212 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8bce09-34ce-4592-889b-d6979cdb36ee None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:25.122785 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8bce09-34ce-4592-889b-d6979cdb36ee None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:25.123127 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8bce09-34ce-4592-889b-d6979cdb36ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:25.161071 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-ea8bce09-34ce-4592-889b-d6979cdb36ee None None] MFA Rules not processed for user `d376a81b81cc45a6b9b1b5f4000e6d75`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:19:25.163427 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-ea8bce09-34ce-4592-889b-d6979cdb36ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:19:25.170235 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-77180801-259b-46a7-b1ed-72596aa82837 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '03d5985494c34df19df17a31fe357d03', 'request_id': 'req-77180801-259b-46a7-b1ed-72596aa82837'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:19:25.179534 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-77180801-259b-46a7-b1ed-72596aa82837 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '03d5985494c34df19df17a31fe357d03', 'request_id': 'req-77180801-259b-46a7-b1ed-72596aa82837'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:19:25.186836 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-77180801-259b-46a7-b1ed-72596aa82837 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '03d5985494c34df19df17a31fe357d03', 'request_id': 'req-77180801-259b-46a7-b1ed-72596aa82837'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:19:25.203058 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-77180801-259b-46a7-b1ed-72596aa82837 admin admin] The token cache is being invalidate because project 03d5985494c34df19df17a31fe357d03 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80381) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 09:19:25.203380 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-77180801-259b-46a7-b1ed-72596aa82837 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:19:25.204579 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-77180801-259b-46a7-b1ed-72596aa82837 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:19:25.211227 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1058/2138] 10.222.0.26 () {58 vars in 1232 bytes} [Thu Jul 14 09:19:25 2022] DELETE /identity/v3/projects/03d5985494c34df19df17a31fe357d03 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:19:25.213642 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040d93fe-169a-47f0-a979-b42b988853ff None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:25.213764 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040d93fe-169a-47f0-a979-b42b988853ff None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:25.213977 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040d93fe-169a-47f0-a979-b42b988853ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:25.236551 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-040d93fe-169a-47f0-a979-b42b988853ff None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:19:25.237761 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-040d93fe-169a-47f0-a979-b42b988853ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:19:25.240316 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1081/2139] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:19:25 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:19:25.254187 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8bce09-34ce-4592-889b-d6979cdb36ee None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:19:25.255943 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-ea8bce09-34ce-4592-889b-d6979cdb36ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:19:25.318155 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1059/2140] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:19:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:19:25.325866 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8bce09-34ce-4592-889b-d6979cdb36ee None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:19:25.329094 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc42de65-bb15-4206-810d-4821e9bdd789 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:25.329549 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc42de65-bb15-4206-810d-4821e9bdd789 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:25.329918 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc42de65-bb15-4206-810d-4821e9bdd789 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:25.330034 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3c0533-f218-415e-8c09-d19da4e74bb7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e76f1952a5b146808844b9f2b7395035', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:19:25.330653 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3c0533-f218-415e-8c09-d19da4e74bb7 service cinder] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:25.330949 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3c0533-f218-415e-8c09-d19da4e74bb7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:25.331251 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3c0533-f218-415e-8c09-d19da4e74bb7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:25.339928 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:19:25.342932 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3c0533-f218-415e-8c09-d19da4e74bb7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:19:25.350892 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3c0533-f218-415e-8c09-d19da4e74bb7 service cinder] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:19:25.355118 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-dc42de65-bb15-4206-810d-4821e9bdd789 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:19:25.356448 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-dc42de65-bb15-4206-810d-4821e9bdd789 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:19:25.369406 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1060/2141] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:19:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:19:25.370285 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1082/2142] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:19:25 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 09:19:25.374138 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc42de65-bb15-4206-810d-4821e9bdd789 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:19:25.384120 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc42de65-bb15-4206-810d-4821e9bdd789 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:19:25.389432 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc644e1d-fc9a-4738-90fb-8669f7cefb84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:19:25.390104 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc644e1d-fc9a-4738-90fb-8669f7cefb84 admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:25.390433 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc644e1d-fc9a-4738-90fb-8669f7cefb84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:25.390714 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc644e1d-fc9a-4738-90fb-8669f7cefb84 admin admin] PATH_INFO: `/v3/domains` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:25.399887 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc644e1d-fc9a-4738-90fb-8669f7cefb84 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:19:25.404637 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc644e1d-fc9a-4738-90fb-8669f7cefb84 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:19:25.407167 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc644e1d-fc9a-4738-90fb-8669f7cefb84 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:19:25.430683 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1061/2143] 10.222.0.26 () {58 vars in 1152 bytes} [Thu Jul 14 09:19:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:19:25.434939 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3c0533-f218-415e-8c09-d19da4e74bb7 service cinder] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:19:25.450907 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3c0533-f218-415e-8c09-d19da4e74bb7 service cinder] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:19:25.458167 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15703690-05f4-4ab6-987f-4fcb22c4297e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:19:25.459542 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15703690-05f4-4ab6-987f-4fcb22c4297e admin admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:25.459542 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15703690-05f4-4ab6-987f-4fcb22c4297e admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:25.460150 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15703690-05f4-4ab6-987f-4fcb22c4297e admin admin] PATH_INFO: `/v3/projects` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:25.462214 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15703690-05f4-4ab6-987f-4fcb22c4297e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:19:25.463476 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15703690-05f4-4ab6-987f-4fcb22c4297e admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:19:25.510678 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1083/2144] 10.222.0.26 () {60 vars in 1152 bytes} [Thu Jul 14 09:19:25 2022] POST /identity/v3/projects => generated 379 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 09:19:25.515860 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc644e1d-fc9a-4738-90fb-8669f7cefb84 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:19:25.526688 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc644e1d-fc9a-4738-90fb-8669f7cefb84 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:19:25.537738 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:19:25.593147 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f17a78-a07a-4985-a558-b792dc8d2832 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:19:25.593879 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f17a78-a07a-4985-a558-b792dc8d2832 admin admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:25.594097 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f17a78-a07a-4985-a558-b792dc8d2832 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:25.594371 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f17a78-a07a-4985-a558-b792dc8d2832 admin admin] PATH_INFO: `/v3/users` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:25.595106 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f17a78-a07a-4985-a558-b792dc8d2832 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:19:25.597318 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f17a78-a07a-4985-a558-b792dc8d2832 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:19:25.625660 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1062/2145] 10.222.0.26 () {60 vars in 1143 bytes} [Thu Jul 14 09:19:25 2022] POST /identity/v3/users => generated 290 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 09:19:25.629888 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15703690-05f4-4ab6-987f-4fcb22c4297e admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:19:25.641794 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15703690-05f4-4ab6-987f-4fcb22c4297e admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:19:25.649536 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24c44d7-99c2-421a-ad6c-6590f478be4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:19:25.650226 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24c44d7-99c2-421a-ad6c-6590f478be4f admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:25.650515 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24c44d7-99c2-421a-ad6c-6590f478be4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:25.650810 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24c44d7-99c2-421a-ad6c-6590f478be4f admin admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:25.651573 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24c44d7-99c2-421a-ad6c-6590f478be4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:19:25.653041 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358a0625-500f-4ee6-9502-3738287fd0a8 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:25.653257 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24c44d7-99c2-421a-ad6c-6590f478be4f admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:19:25.653830 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358a0625-500f-4ee6-9502-3738287fd0a8 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:25.654371 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358a0625-500f-4ee6-9502-3738287fd0a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:25.667643 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1084/2146] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jul 14 09:19:25 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 09:19:25.671834 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24c44d7-99c2-421a-ad6c-6590f478be4f admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:19:25.677997 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-358a0625-500f-4ee6-9502-3738287fd0a8 None None] MFA Rules not processed for user `7d6267ede8d64e2abe1f4058a9797fb7`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:19:25.680071 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-358a0625-500f-4ee6-9502-3738287fd0a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:19:25.683757 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24c44d7-99c2-421a-ad6c-6590f478be4f admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:19:25.689815 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788bfd66-fda1-4e92-858a-6f724fc7813d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:19:25.690714 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788bfd66-fda1-4e92-858a-6f724fc7813d admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:25.691755 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788bfd66-fda1-4e92-858a-6f724fc7813d admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:25.692530 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788bfd66-fda1-4e92-858a-6f724fc7813d admin admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:25.693029 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788bfd66-fda1-4e92-858a-6f724fc7813d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:19:25.694703 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788bfd66-fda1-4e92-858a-6f724fc7813d admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:19:25.706022 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1085/2147] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jul 14 09:19:25 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 09:19:25.712194 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788bfd66-fda1-4e92-858a-6f724fc7813d admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:19:25.733891 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788bfd66-fda1-4e92-858a-6f724fc7813d admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:19:25.738762 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8dcfa4a-0519-409c-96b9-39bae87e2b6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:19:25.739692 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dcfa4a-0519-409c-96b9-39bae87e2b6e admin admin] REQUEST_METHOD: `PUT` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:25.739965 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dcfa4a-0519-409c-96b9-39bae87e2b6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:25.740233 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dcfa4a-0519-409c-96b9-39bae87e2b6e admin admin] PATH_INFO: `/v3/projects/132e13bb6b1c413a8656f7a4ad816232/users/e072ca213ed74cbda810a1fd971ae23d/roles/d1691322ce6a4edab7ce722edf0f2988` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:25.762538 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8dcfa4a-0519-409c-96b9-39bae87e2b6e admin admin] RBAC: Authorizing `identity:create_grant(project_id=132e13bb6b1c413a8656f7a4ad816232, user_id=e072ca213ed74cbda810a1fd971ae23d, role_id=d1691322ce6a4edab7ce722edf0f2988)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:19:25.765357 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8dcfa4a-0519-409c-96b9-39bae87e2b6e admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:19:25.784431 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1063/2148] 10.222.0.26 () {58 vars in 967 bytes} [Thu Jul 14 09:19:25 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 09:19:25.797506 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1086/2149] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jul 14 09:19:25 2022] PUT /identity/v3/projects/132e13bb6b1c413a8656f7a4ad816232/users/e072ca213ed74cbda810a1fd971ae23d/roles/d1691322ce6a4edab7ce722edf0f2988 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:19:25.798777 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358a0625-500f-4ee6-9502-3738287fd0a8 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:19:25.800544 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-358a0625-500f-4ee6-9502-3738287fd0a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:19:25.807562 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3471c2-2805-4a7c-b361-81cdad78196e None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:25.807853 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3471c2-2805-4a7c-b361-81cdad78196e None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:25.808134 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3471c2-2805-4a7c-b361-81cdad78196e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:25.826099 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-1c3471c2-2805-4a7c-b361-81cdad78196e None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:19:25.827417 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-1c3471c2-2805-4a7c-b361-81cdad78196e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:19:25.880668 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358a0625-500f-4ee6-9502-3738287fd0a8 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:19:25.891891 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1087/2150] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:19:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:19:25.898774 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3471c2-2805-4a7c-b361-81cdad78196e None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:19:25.900958 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-1c3471c2-2805-4a7c-b361-81cdad78196e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:19:25.955697 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:19:25.971816 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea76cdce-d22e-44c1-a5eb-72ead2ef0b04 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5a43c80514b4a13af03c7f8ebcf7a5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:19:25.972606 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea76cdce-d22e-44c1-a5eb-72ead2ef0b04 service swift] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:25.972931 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea76cdce-d22e-44c1-a5eb-72ead2ef0b04 service swift] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:25.973453 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea76cdce-d22e-44c1-a5eb-72ead2ef0b04 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:25.978547 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3471c2-2805-4a7c-b361-81cdad78196e None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:19:25.983411 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a09ad64-a4fc-47aa-a4f7-51a0008926d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '952911c492444ffea4b211a24b564117', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:19:25.987043 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a09ad64-a4fc-47aa-a4f7-51a0008926d0 service neutron] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:25.987043 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a09ad64-a4fc-47aa-a4f7-51a0008926d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:25.987043 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a09ad64-a4fc-47aa-a4f7-51a0008926d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:25.997265 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a09ad64-a4fc-47aa-a4f7-51a0008926d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:19:25.998784 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a09ad64-a4fc-47aa-a4f7-51a0008926d0 service neutron] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:19:26.011587 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1088/2151] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:19:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 09:19:26.042916 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea76cdce-d22e-44c1-a5eb-72ead2ef0b04 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:19:26.044796 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea76cdce-d22e-44c1-a5eb-72ead2ef0b04 service swift] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:19:26.055678 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1064/2152] 10.222.0.26 () {60 vars in 1384 bytes} [Thu Jul 14 09:19:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 258 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 14 09:19:26.273233 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d3aa3f-e262-4abb-8aa6-5268a1a25583 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:26.273647 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d3aa3f-e262-4abb-8aa6-5268a1a25583 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:26.274017 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d3aa3f-e262-4abb-8aa6-5268a1a25583 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:26.295005 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-a9d3aa3f-e262-4abb-8aa6-5268a1a25583 None None] MFA Rules not processed for user `7d6267ede8d64e2abe1f4058a9797fb7`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:19:26.296511 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-a9d3aa3f-e262-4abb-8aa6-5268a1a25583 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:19:26.304039 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1089/2153] 10.222.0.26 () {58 vars in 1009 bytes} [Thu Jul 14 09:19:26 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:19:26.317063 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea76cdce-d22e-44c1-a5eb-72ead2ef0b04 service swift] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:19:26.328847 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea76cdce-d22e-44c1-a5eb-72ead2ef0b04 service swift] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:19:26.334089 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2313498-fa78-4daa-ad23-6c9e83349f35 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5a43c80514b4a13af03c7f8ebcf7a5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:19:26.334737 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2313498-fa78-4daa-ad23-6c9e83349f35 service swift] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:26.335237 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2313498-fa78-4daa-ad23-6c9e83349f35 service swift] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:26.335353 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2313498-fa78-4daa-ad23-6c9e83349f35 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:26.344882 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2313498-fa78-4daa-ad23-6c9e83349f35 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:19:26.346554 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2313498-fa78-4daa-ad23-6c9e83349f35 service swift] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:19:26.355614 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1065/2154] 10.222.0.26 () {60 vars in 1384 bytes} [Thu Jul 14 09:19:26 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 14 09:19:26.732233 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d602c427-a4ee-40ee-8687-917ffe23face None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:26.732721 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d602c427-a4ee-40ee-8687-917ffe23face None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:26.732933 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d602c427-a4ee-40ee-8687-917ffe23face None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:26.752428 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-d602c427-a4ee-40ee-8687-917ffe23face None None] MFA Rules not processed for user `7d6267ede8d64e2abe1f4058a9797fb7`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:19:26.753637 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-d602c427-a4ee-40ee-8687-917ffe23face None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:19:26.761960 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1090/2155] 10.222.0.26 () {58 vars in 967 bytes} [Thu Jul 14 09:19:26 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:19:26.776128 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2313498-fa78-4daa-ad23-6c9e83349f35 service swift] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:19:26.795325 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2313498-fa78-4daa-ad23-6c9e83349f35 service swift] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:19:26.801934 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7559c13b-512b-4462-828d-8027f6baa413 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5a43c80514b4a13af03c7f8ebcf7a5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:19:26.802600 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7559c13b-512b-4462-828d-8027f6baa413 service swift] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:26.802888 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7559c13b-512b-4462-828d-8027f6baa413 service swift] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:26.804897 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7559c13b-512b-4462-828d-8027f6baa413 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:26.814962 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7559c13b-512b-4462-828d-8027f6baa413 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:19:26.817085 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7559c13b-512b-4462-828d-8027f6baa413 service swift] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:19:26.847707 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1066/2156] 10.222.0.26 () {60 vars in 1384 bytes} [Thu Jul 14 09:19:26 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 14 09:19:27.071504 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d602c427-a4ee-40ee-8687-917ffe23face None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:19:27.073008 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-d602c427-a4ee-40ee-8687-917ffe23face None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:19:27.096990 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d602c427-a4ee-40ee-8687-917ffe23face None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:19:27.106642 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8c52e9-01bb-4717-bd85-768b02c4ff99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:19:27.107868 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8c52e9-01bb-4717-bd85-768b02c4ff99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:27.110378 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8c52e9-01bb-4717-bd85-768b02c4ff99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:27.110765 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8c52e9-01bb-4717-bd85-768b02c4ff99 admin admin] PATH_INFO: `/v3/users/ed3ee4b41b6e437a9e23a53be94f1455` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:27.115360 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8c52e9-01bb-4717-bd85-768b02c4ff99 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ed3ee4b41b6e437a9e23a53be94f1455)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:19:27.117437 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8c52e9-01bb-4717-bd85-768b02c4ff99 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:19:27.145805 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:DELETE=1 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:19:27.217248 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-0b8c52e9-01bb-4717-bd85-768b02c4ff99 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ed3ee4b41b6e437a9e23a53be94f1455'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:19:27.219272 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-0b8c52e9-01bb-4717-bd85-768b02c4ff99 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ed3ee4b41b6e437a9e23a53be94f1455'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:19:27.260152 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-0b8c52e9-01bb-4717-bd85-768b02c4ff99 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ed3ee4b41b6e437a9e23a53be94f1455'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:19:27.275299 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-0b8c52e9-01bb-4717-bd85-768b02c4ff99 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ed3ee4b41b6e437a9e23a53be94f1455'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:19:27.276160 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-0b8c52e9-01bb-4717-bd85-768b02c4ff99 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ed3ee4b41b6e437a9e23a53be94f1455'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:19:27.298266 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffe4362-ea9a-4d6b-8cd7-09a86843259a None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:27.298664 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffe4362-ea9a-4d6b-8cd7-09a86843259a None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:27.299037 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffe4362-ea9a-4d6b-8cd7-09a86843259a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:27.300675 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1091/2157] 10.222.0.26 () {58 vars in 1223 bytes} [Thu Jul 14 09:19:27 2022] DELETE /identity/v3/users/ed3ee4b41b6e437a9e23a53be94f1455 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:19:27.334670 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-fffe4362-ea9a-4d6b-8cd7-09a86843259a None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:19:27.337010 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-fffe4362-ea9a-4d6b-8cd7-09a86843259a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:19:27.476203 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1067/2158] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:19:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:19:27.484094 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8c52e9-01bb-4717-bd85-768b02c4ff99 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:19:27.484094 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-0b8c52e9-01bb-4717-bd85-768b02c4ff99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:19:27.604837 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8c52e9-01bb-4717-bd85-768b02c4ff99 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:19:27.631427 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c35c7b-8def-4814-8235-7fa1d3b0a780 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '952911c492444ffea4b211a24b564117', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:19:27.632245 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c35c7b-8def-4814-8235-7fa1d3b0a780 service neutron] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:27.632588 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c35c7b-8def-4814-8235-7fa1d3b0a780 service neutron] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:27.632936 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c35c7b-8def-4814-8235-7fa1d3b0a780 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:27.658229 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c35c7b-8def-4814-8235-7fa1d3b0a780 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:19:27.660739 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c35c7b-8def-4814-8235-7fa1d3b0a780 service neutron] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:19:27.678988 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1092/2159] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:19:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:19:28.280690 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffe4362-ea9a-4d6b-8cd7-09a86843259a None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:19:28.282225 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-fffe4362-ea9a-4d6b-8cd7-09a86843259a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:19:28.301095 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffe4362-ea9a-4d6b-8cd7-09a86843259a None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:19:28.306992 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bb67a5-01e9-4614-8f2e-411de09f1054 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:19:28.307938 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bb67a5-01e9-4614-8f2e-411de09f1054 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:28.308222 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bb67a5-01e9-4614-8f2e-411de09f1054 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:28.308504 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bb67a5-01e9-4614-8f2e-411de09f1054 admin admin] PATH_INFO: `/v3/projects/a09284b0d51f4148ace0c3cf1b58d266` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:28.309957 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bb67a5-01e9-4614-8f2e-411de09f1054 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a09284b0d51f4148ace0c3cf1b58d266)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:19:28.313583 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bb67a5-01e9-4614-8f2e-411de09f1054 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:19:28.392343 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-96bb67a5-01e9-4614-8f2e-411de09f1054 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a09284b0d51f4148ace0c3cf1b58d266', 'request_id': 'req-96bb67a5-01e9-4614-8f2e-411de09f1054'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:19:28.393242 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c35c7b-8def-4814-8235-7fa1d3b0a780 service neutron] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:19:28.395472 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-39c35c7b-8def-4814-8235-7fa1d3b0a780 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:19:28.406689 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-96bb67a5-01e9-4614-8f2e-411de09f1054 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a09284b0d51f4148ace0c3cf1b58d266', 'request_id': 'req-96bb67a5-01e9-4614-8f2e-411de09f1054'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:19:28.410949 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c35c7b-8def-4814-8235-7fa1d3b0a780 service neutron] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:19:28.413452 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:19:28.416844 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-96bb67a5-01e9-4614-8f2e-411de09f1054 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a09284b0d51f4148ace0c3cf1b58d266', 'request_id': 'req-96bb67a5-01e9-4614-8f2e-411de09f1054'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:19:28.451130 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-96bb67a5-01e9-4614-8f2e-411de09f1054 admin admin] The token cache is being invalidate because project a09284b0d51f4148ace0c3cf1b58d266 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80381) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 09:19:28.451130 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-96bb67a5-01e9-4614-8f2e-411de09f1054 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:19:28.451130 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-96bb67a5-01e9-4614-8f2e-411de09f1054 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:19:28.456949 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1068/2160] 10.222.0.26 () {58 vars in 1232 bytes} [Thu Jul 14 09:19:28 2022] DELETE /identity/v3/projects/a09284b0d51f4148ace0c3cf1b58d266 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:19:28.465249 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ba4a99-e570-420e-8948-5a44d02492d0 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:28.465752 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ba4a99-e570-420e-8948-5a44d02492d0 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:28.466744 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ba4a99-e570-420e-8948-5a44d02492d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:28.495570 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-20ba4a99-e570-420e-8948-5a44d02492d0 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:19:28.497138 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-20ba4a99-e570-420e-8948-5a44d02492d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:19:28.587524 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09505c29-ce00-4ad1-8744-1a18d8874be6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:19:28.588719 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09505c29-ce00-4ad1-8744-1a18d8874be6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:28.589224 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09505c29-ce00-4ad1-8744-1a18d8874be6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:28.589691 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09505c29-ce00-4ad1-8744-1a18d8874be6 admin admin] PATH_INFO: `/v3/users/d376a81b81cc45a6b9b1b5f4000e6d75` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:28.591431 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09505c29-ce00-4ad1-8744-1a18d8874be6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d376a81b81cc45a6b9b1b5f4000e6d75)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:19:28.594252 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09505c29-ce00-4ad1-8744-1a18d8874be6 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:19:28.686077 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:19:28.690474 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1069/2161] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:19:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:19:28.700218 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f776f1-6b57-4747-bd87-5dc0fb4f7e9a None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:28.702569 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f776f1-6b57-4747-bd87-5dc0fb4f7e9a None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:28.702962 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f776f1-6b57-4747-bd87-5dc0fb4f7e9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:28.731700 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-09505c29-ce00-4ad1-8744-1a18d8874be6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd376a81b81cc45a6b9b1b5f4000e6d75'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:19:28.732447 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-09505c29-ce00-4ad1-8744-1a18d8874be6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd376a81b81cc45a6b9b1b5f4000e6d75'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:19:28.742356 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-17f776f1-6b57-4747-bd87-5dc0fb4f7e9a None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:19:28.743905 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-17f776f1-6b57-4747-bd87-5dc0fb4f7e9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:19:28.752781 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1070/2162] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:19:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:19:28.757946 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f776f1-6b57-4747-bd87-5dc0fb4f7e9a None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:19:28.768554 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-09505c29-ce00-4ad1-8744-1a18d8874be6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd376a81b81cc45a6b9b1b5f4000e6d75'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:19:28.773136 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f776f1-6b57-4747-bd87-5dc0fb4f7e9a None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:19:28.779405 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bec788-47f3-4d5c-a06c-19780be3c972 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:19:28.780288 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bec788-47f3-4d5c-a06c-19780be3c972 admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:28.780674 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bec788-47f3-4d5c-a06c-19780be3c972 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:28.781387 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bec788-47f3-4d5c-a06c-19780be3c972 admin admin] PATH_INFO: `/v3/domains` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:28.782797 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bec788-47f3-4d5c-a06c-19780be3c972 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:19:28.783565 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bec788-47f3-4d5c-a06c-19780be3c972 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:19:28.785361 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-09505c29-ce00-4ad1-8744-1a18d8874be6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd376a81b81cc45a6b9b1b5f4000e6d75'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:19:28.786003 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-09505c29-ce00-4ad1-8744-1a18d8874be6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd376a81b81cc45a6b9b1b5f4000e6d75'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:19:28.786427 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bec788-47f3-4d5c-a06c-19780be3c972 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:19:28.803493 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1093/2163] 10.222.0.26 () {58 vars in 1223 bytes} [Thu Jul 14 09:19:28 2022] DELETE /identity/v3/users/d376a81b81cc45a6b9b1b5f4000e6d75 => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:19:28.805341 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1071/2164] 10.222.0.26 () {58 vars in 1152 bytes} [Thu Jul 14 09:19:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 09:19:28.806533 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd2bfa5-ede9-4b94-935a-6c0800dfef75 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:28.806827 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd2bfa5-ede9-4b94-935a-6c0800dfef75 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:28.807179 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd2bfa5-ede9-4b94-935a-6c0800dfef75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:28.808529 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bec788-47f3-4d5c-a06c-19780be3c972 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:19:28.820805 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bec788-47f3-4d5c-a06c-19780be3c972 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:19:28.842269 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-8dd2bfa5-ede9-4b94-935a-6c0800dfef75 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:19:28.843709 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-8dd2bfa5-ede9-4b94-935a-6c0800dfef75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:19:28.932235 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a66343-dfa5-4c3f-94c2-b59a35fe08ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:19:28.932918 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a66343-dfa5-4c3f-94c2-b59a35fe08ae admin admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:28.933206 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a66343-dfa5-4c3f-94c2-b59a35fe08ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:28.933487 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a66343-dfa5-4c3f-94c2-b59a35fe08ae admin admin] PATH_INFO: `/v3/projects` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:28.938620 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a66343-dfa5-4c3f-94c2-b59a35fe08ae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:19:28.945051 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a66343-dfa5-4c3f-94c2-b59a35fe08ae admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:19:28.988454 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1094/2165] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:19:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 09:19:28.996973 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1072/2166] 10.222.0.26 () {60 vars in 1152 bytes} [Thu Jul 14 09:19:28 2022] POST /identity/v3/projects => generated 391 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 09:19:29.001352 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd2bfa5-ede9-4b94-935a-6c0800dfef75 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:19:29.004444 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a66343-dfa5-4c3f-94c2-b59a35fe08ae admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:19:29.016798 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-53a66343-dfa5-4c3f-94c2-b59a35fe08ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:19:29.018171 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd2bfa5-ede9-4b94-935a-6c0800dfef75 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:19:29.115027 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c903c5f2-98b0-49e7-9813-1a597d77dfc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:19:29.115843 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c903c5f2-98b0-49e7-9813-1a597d77dfc2 admin admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:29.116130 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c903c5f2-98b0-49e7-9813-1a597d77dfc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:29.116407 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c903c5f2-98b0-49e7-9813-1a597d77dfc2 admin admin] PATH_INFO: `/v3/users` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:29.117115 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c903c5f2-98b0-49e7-9813-1a597d77dfc2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:19:29.119049 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c903c5f2-98b0-49e7-9813-1a597d77dfc2 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:19:29.122642 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a66343-dfa5-4c3f-94c2-b59a35fe08ae admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:19:29.126518 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f197e6d-4cc9-4f6a-a43e-e6977e4ea4d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '952911c492444ffea4b211a24b564117', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:19:29.127412 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f197e6d-4cc9-4f6a-a43e-e6977e4ea4d5 service neutron] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:29.127777 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f197e6d-4cc9-4f6a-a43e-e6977e4ea4d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:29.128735 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f197e6d-4cc9-4f6a-a43e-e6977e4ea4d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:29.139522 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:DELETE=2,keystone:SELECT=96,keystone:INSERT=2 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:19:29.143138 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f197e6d-4cc9-4f6a-a43e-e6977e4ea4d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:19:29.145333 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f197e6d-4cc9-4f6a-a43e-e6977e4ea4d5 service neutron] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:19:29.148815 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1095/2167] 10.222.0.26 () {60 vars in 1143 bytes} [Thu Jul 14 09:19:29 2022] POST /identity/v3/users => generated 296 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 09:19:29.152706 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c903c5f2-98b0-49e7-9813-1a597d77dfc2 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:19:29.164342 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c903c5f2-98b0-49e7-9813-1a597d77dfc2 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:19:29.171329 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b3bd0c-a5ed-4932-9d4e-eb8562f88195 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:19:29.171499 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b3bd0c-a5ed-4932-9d4e-eb8562f88195 admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:29.172138 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b3bd0c-a5ed-4932-9d4e-eb8562f88195 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:29.172435 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b3bd0c-a5ed-4932-9d4e-eb8562f88195 admin admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:29.173276 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b3bd0c-a5ed-4932-9d4e-eb8562f88195 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:19:29.175164 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b3bd0c-a5ed-4932-9d4e-eb8562f88195 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:19:29.180105 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1073/2168] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:19:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:19:29.186864 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1096/2169] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jul 14 09:19:29 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 09:19:29.190988 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f197e6d-4cc9-4f6a-a43e-e6977e4ea4d5 service neutron] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:19:29.201751 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f197e6d-4cc9-4f6a-a43e-e6977e4ea4d5 service neutron] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:19:29.206418 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac349d9-bb11-4097-95c8-acbf643c96e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:19:29.207087 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac349d9-bb11-4097-95c8-acbf643c96e2 admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:29.207401 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac349d9-bb11-4097-95c8-acbf643c96e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:29.207679 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac349d9-bb11-4097-95c8-acbf643c96e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:29.208409 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac349d9-bb11-4097-95c8-acbf643c96e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:19:29.210202 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac349d9-bb11-4097-95c8-acbf643c96e2 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:19:29.223244 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1074/2170] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jul 14 09:19:29 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 09:19:29.226337 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b3bd0c-a5ed-4932-9d4e-eb8562f88195 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:19:29.239969 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b3bd0c-a5ed-4932-9d4e-eb8562f88195 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:19:29.248036 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938a9a22-114f-4b74-a449-3a00bc4d4941 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:19:29.250607 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938a9a22-114f-4b74-a449-3a00bc4d4941 admin admin] REQUEST_METHOD: `PUT` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:29.253278 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938a9a22-114f-4b74-a449-3a00bc4d4941 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:29.253733 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938a9a22-114f-4b74-a449-3a00bc4d4941 admin admin] PATH_INFO: `/v3/projects/2abd5167cfff4a66b972cda4b9fb8b4f/users/8baac06bf33a4f2182a30917293666ac/roles/d1691322ce6a4edab7ce722edf0f2988` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:29.292977 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938a9a22-114f-4b74-a449-3a00bc4d4941 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2abd5167cfff4a66b972cda4b9fb8b4f, user_id=8baac06bf33a4f2182a30917293666ac, role_id=d1691322ce6a4edab7ce722edf0f2988)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:19:29.295913 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938a9a22-114f-4b74-a449-3a00bc4d4941 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:19:29.315250 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1097/2171] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jul 14 09:19:29 2022] PUT /identity/v3/projects/2abd5167cfff4a66b972cda4b9fb8b4f/users/8baac06bf33a4f2182a30917293666ac/roles/d1691322ce6a4edab7ce722edf0f2988 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:19:29.325789 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a952e547-822a-496d-a0b3-ed109ca775f8 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:29.326298 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a952e547-822a-496d-a0b3-ed109ca775f8 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:29.326613 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a952e547-822a-496d-a0b3-ed109ca775f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:29.361909 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-a952e547-822a-496d-a0b3-ed109ca775f8 None None] MFA Rules not processed for user `8baac06bf33a4f2182a30917293666ac`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:19:29.363393 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-a952e547-822a-496d-a0b3-ed109ca775f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:19:29.467706 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1075/2172] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:19:29 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:19:29.481306 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938a9a22-114f-4b74-a449-3a00bc4d4941 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:19:29.484904 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-938a9a22-114f-4b74-a449-3a00bc4d4941 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:19:29.505626 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:19:29.567040 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a952e547-822a-496d-a0b3-ed109ca775f8 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:19:29.567720 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-a952e547-822a-496d-a0b3-ed109ca775f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:19:29.591391 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938a9a22-114f-4b74-a449-3a00bc4d4941 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:19:29.595382 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef23db3-4ca9-4b19-aa47-e82a3ff4d8b2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e76f1952a5b146808844b9f2b7395035', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:19:29.596247 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef23db3-4ca9-4b19-aa47-e82a3ff4d8b2 service cinder] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:29.596602 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef23db3-4ca9-4b19-aa47-e82a3ff4d8b2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:29.598158 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef23db3-4ca9-4b19-aa47-e82a3ff4d8b2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:29.609681 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef23db3-4ca9-4b19-aa47-e82a3ff4d8b2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:19:29.611293 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef23db3-4ca9-4b19-aa47-e82a3ff4d8b2 service cinder] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:19:29.622951 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1098/2173] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:19:29 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:19:29.651362 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a952e547-822a-496d-a0b3-ed109ca775f8 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:19:29.656433 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34ab7a9-480d-45bb-ad73-01ad0c5ecd56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:19:29.657408 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34ab7a9-480d-45bb-ad73-01ad0c5ecd56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:29.657695 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34ab7a9-480d-45bb-ad73-01ad0c5ecd56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:29.658079 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34ab7a9-480d-45bb-ad73-01ad0c5ecd56 admin admin] PATH_INFO: `/v3/projects/2392e639b4814c6a9783bad07d2b1afb` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:29.659697 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34ab7a9-480d-45bb-ad73-01ad0c5ecd56 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2392e639b4814c6a9783bad07d2b1afb)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:19:29.671229 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34ab7a9-480d-45bb-ad73-01ad0c5ecd56 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:19:29.750050 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-d34ab7a9-480d-45bb-ad73-01ad0c5ecd56 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2392e639b4814c6a9783bad07d2b1afb', 'request_id': 'req-d34ab7a9-480d-45bb-ad73-01ad0c5ecd56'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:19:29.769299 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-d34ab7a9-480d-45bb-ad73-01ad0c5ecd56 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2392e639b4814c6a9783bad07d2b1afb', 'request_id': 'req-d34ab7a9-480d-45bb-ad73-01ad0c5ecd56'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:19:29.770425 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef23db3-4ca9-4b19-aa47-e82a3ff4d8b2 service cinder] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:19:29.772197 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-cef23db3-4ca9-4b19-aa47-e82a3ff4d8b2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:19:29.789956 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-d34ab7a9-480d-45bb-ad73-01ad0c5ecd56 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2392e639b4814c6a9783bad07d2b1afb', 'request_id': 'req-d34ab7a9-480d-45bb-ad73-01ad0c5ecd56'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:19:29.797245 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef23db3-4ca9-4b19-aa47-e82a3ff4d8b2 service cinder] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:19:29.826193 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-d34ab7a9-480d-45bb-ad73-01ad0c5ecd56 admin admin] The token cache is being invalidate because project 2392e639b4814c6a9783bad07d2b1afb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80381) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 09:19:29.826770 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-d34ab7a9-480d-45bb-ad73-01ad0c5ecd56 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:19:29.829208 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-d34ab7a9-480d-45bb-ad73-01ad0c5ecd56 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:19:29.833814 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1076/2174] 10.222.0.26 () {58 vars in 1232 bytes} [Thu Jul 14 09:19:29 2022] DELETE /identity/v3/projects/2392e639b4814c6a9783bad07d2b1afb => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:19:29.839072 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f017ceb-bcd7-4747-8ea1-4f0e8381d411 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:29.840619 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f017ceb-bcd7-4747-8ea1-4f0e8381d411 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:29.841744 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f017ceb-bcd7-4747-8ea1-4f0e8381d411 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:29.856409 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:19:29.878646 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-8f017ceb-bcd7-4747-8ea1-4f0e8381d411 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:19:29.881473 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-8f017ceb-bcd7-4747-8ea1-4f0e8381d411 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:19:29.940225 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcdd572-9c96-4f8d-94ed-2b360b68d3e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:19:29.941354 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcdd572-9c96-4f8d-94ed-2b360b68d3e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:29.945124 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcdd572-9c96-4f8d-94ed-2b360b68d3e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:29.945843 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcdd572-9c96-4f8d-94ed-2b360b68d3e1 admin admin] PATH_INFO: `/v3/users/38970dab4776492a8f2fccaf9705ebc0` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:29.948054 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcdd572-9c96-4f8d-94ed-2b360b68d3e1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=38970dab4776492a8f2fccaf9705ebc0)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:19:29.960230 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcdd572-9c96-4f8d-94ed-2b360b68d3e1 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:19:30.046149 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1077/2175] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:19:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 09:19:30.066594 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24eb566-9be3-4dd0-b864-96d407a6660a None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:30.066594 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24eb566-9be3-4dd0-b864-96d407a6660a None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:30.066594 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24eb566-9be3-4dd0-b864-96d407a6660a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:30.088689 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-0fcdd572-9c96-4f8d-94ed-2b360b68d3e1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '38970dab4776492a8f2fccaf9705ebc0'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:19:30.089614 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-0fcdd572-9c96-4f8d-94ed-2b360b68d3e1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '38970dab4776492a8f2fccaf9705ebc0'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:19:30.092447 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-f24eb566-9be3-4dd0-b864-96d407a6660a None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:19:30.095876 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-f24eb566-9be3-4dd0-b864-96d407a6660a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:19:30.130655 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-0fcdd572-9c96-4f8d-94ed-2b360b68d3e1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '38970dab4776492a8f2fccaf9705ebc0'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:19:30.135351 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1078/2176] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:19:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 09:19:30.143611 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24eb566-9be3-4dd0-b864-96d407a6660a None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:19:30.153757 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-0fcdd572-9c96-4f8d-94ed-2b360b68d3e1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '38970dab4776492a8f2fccaf9705ebc0'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:19:30.157132 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-0fcdd572-9c96-4f8d-94ed-2b360b68d3e1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '38970dab4776492a8f2fccaf9705ebc0'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:19:30.173892 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24eb566-9be3-4dd0-b864-96d407a6660a None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:19:30.179340 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1099/2177] 10.222.0.26 () {58 vars in 1223 bytes} [Thu Jul 14 09:19:29 2022] DELETE /identity/v3/users/38970dab4776492a8f2fccaf9705ebc0 => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 09:19:30.188044 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586d518c-945d-4d1d-8fbc-03cc9ee2e3e7 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:30.188339 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586d518c-945d-4d1d-8fbc-03cc9ee2e3e7 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:30.188621 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586d518c-945d-4d1d-8fbc-03cc9ee2e3e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:30.198971 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fb0749-240d-44fa-8d81-b99ccbceac90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:19:30.199782 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fb0749-240d-44fa-8d81-b99ccbceac90 admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:30.200043 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fb0749-240d-44fa-8d81-b99ccbceac90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:30.200409 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fb0749-240d-44fa-8d81-b99ccbceac90 admin admin] PATH_INFO: `/v3/domains` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:30.201224 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fb0749-240d-44fa-8d81-b99ccbceac90 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:19:30.201710 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fb0749-240d-44fa-8d81-b99ccbceac90 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:19:30.203524 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fb0749-240d-44fa-8d81-b99ccbceac90 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:19:30.227933 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-586d518c-945d-4d1d-8fbc-03cc9ee2e3e7 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:19:30.229833 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-586d518c-945d-4d1d-8fbc-03cc9ee2e3e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:19:30.236594 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1079/2178] 10.222.0.26 () {58 vars in 1152 bytes} [Thu Jul 14 09:19:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:19:30.237712 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fb0749-240d-44fa-8d81-b99ccbceac90 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:19:30.256098 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fb0749-240d-44fa-8d81-b99ccbceac90 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:19:30.285351 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:19:30.358759 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1100/2179] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:19:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:19:30.394377 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586d518c-945d-4d1d-8fbc-03cc9ee2e3e7 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:19:30.396424 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-586d518c-945d-4d1d-8fbc-03cc9ee2e3e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:19:30.417660 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436ce6b6-892b-41f2-b5a4-68a5baafd32e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:19:30.419678 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436ce6b6-892b-41f2-b5a4-68a5baafd32e admin admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:30.419951 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436ce6b6-892b-41f2-b5a4-68a5baafd32e admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:30.422428 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436ce6b6-892b-41f2-b5a4-68a5baafd32e admin admin] PATH_INFO: `/v3/projects` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:30.423173 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436ce6b6-892b-41f2-b5a4-68a5baafd32e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:19:30.427370 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436ce6b6-892b-41f2-b5a4-68a5baafd32e admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:19:30.474871 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1080/2180] 10.222.0.26 () {60 vars in 1152 bytes} [Thu Jul 14 09:19:30 2022] POST /identity/v3/projects => generated 371 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 09:19:30.478917 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436ce6b6-892b-41f2-b5a4-68a5baafd32e admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:19:30.579913 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586d518c-945d-4d1d-8fbc-03cc9ee2e3e7 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:19:30.616591 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436ce6b6-892b-41f2-b5a4-68a5baafd32e admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:19:30.639567 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c594a2-2cb9-41ac-b435-25915c6a6bb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:19:30.641776 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c594a2-2cb9-41ac-b435-25915c6a6bb2 admin admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:30.642216 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c594a2-2cb9-41ac-b435-25915c6a6bb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:30.643459 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c594a2-2cb9-41ac-b435-25915c6a6bb2 admin admin] PATH_INFO: `/v3/users` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:30.646900 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c594a2-2cb9-41ac-b435-25915c6a6bb2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:19:30.652620 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c594a2-2cb9-41ac-b435-25915c6a6bb2 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:19:30.653933 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7659570-cbb9-4e57-b44c-42d98badd6e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '952911c492444ffea4b211a24b564117', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:19:30.654609 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7659570-cbb9-4e57-b44c-42d98badd6e1 service neutron] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:30.654945 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7659570-cbb9-4e57-b44c-42d98badd6e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:30.655249 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7659570-cbb9-4e57-b44c-42d98badd6e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:30.678744 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7659570-cbb9-4e57-b44c-42d98badd6e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:19:30.680427 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7659570-cbb9-4e57-b44c-42d98badd6e1 service neutron] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:19:30.694523 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:19:30.695610 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1101/2181] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:19:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 09:19:30.713040 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7659570-cbb9-4e57-b44c-42d98badd6e1 service neutron] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:19:30.713289 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1081/2182] 10.222.0.26 () {60 vars in 1143 bytes} [Thu Jul 14 09:19:30 2022] POST /identity/v3/users => generated 286 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 09:19:30.726707 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7659570-cbb9-4e57-b44c-42d98badd6e1 service neutron] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:19:30.732322 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613a0d3a-c3ad-4936-9737-f23ad0b898e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:19:30.733676 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613a0d3a-c3ad-4936-9737-f23ad0b898e4 admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:30.733962 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613a0d3a-c3ad-4936-9737-f23ad0b898e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:30.734321 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613a0d3a-c3ad-4936-9737-f23ad0b898e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:30.736347 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613a0d3a-c3ad-4936-9737-f23ad0b898e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:19:30.738649 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613a0d3a-c3ad-4936-9737-f23ad0b898e4 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:19:30.752998 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1102/2183] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jul 14 09:19:30 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 09:19:30.754869 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c594a2-2cb9-41ac-b435-25915c6a6bb2 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:19:30.771749 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c594a2-2cb9-41ac-b435-25915c6a6bb2 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:19:30.801551 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5bf156-6f73-4df6-b8c8-956634f5d329 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:19:30.804525 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5bf156-6f73-4df6-b8c8-956634f5d329 admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:30.804968 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5bf156-6f73-4df6-b8c8-956634f5d329 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:30.805380 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5bf156-6f73-4df6-b8c8-956634f5d329 admin admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:30.806368 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5bf156-6f73-4df6-b8c8-956634f5d329 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:19:30.808859 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5bf156-6f73-4df6-b8c8-956634f5d329 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:19:30.825265 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1082/2184] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jul 14 09:19:30 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 09:19:30.826392 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613a0d3a-c3ad-4936-9737-f23ad0b898e4 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:19:30.841856 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613a0d3a-c3ad-4936-9737-f23ad0b898e4 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:19:30.853206 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5393d69d-f9a0-45f4-b92d-afc78ff3d659 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:19:30.854655 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5393d69d-f9a0-45f4-b92d-afc78ff3d659 admin admin] REQUEST_METHOD: `PUT` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:30.855332 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5393d69d-f9a0-45f4-b92d-afc78ff3d659 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:30.855777 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5393d69d-f9a0-45f4-b92d-afc78ff3d659 admin admin] PATH_INFO: `/v3/projects/05d00975363a48f7b9d3efc4aa176f85/users/265fa1d06bc049db92316c145970f887/roles/d1691322ce6a4edab7ce722edf0f2988` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:30.881247 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5393d69d-f9a0-45f4-b92d-afc78ff3d659 admin admin] RBAC: Authorizing `identity:create_grant(project_id=05d00975363a48f7b9d3efc4aa176f85, user_id=265fa1d06bc049db92316c145970f887, role_id=d1691322ce6a4edab7ce722edf0f2988)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:19:30.885991 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5393d69d-f9a0-45f4-b92d-afc78ff3d659 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:19:30.909966 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1103/2185] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jul 14 09:19:30 2022] PUT /identity/v3/projects/05d00975363a48f7b9d3efc4aa176f85/users/265fa1d06bc049db92316c145970f887/roles/d1691322ce6a4edab7ce722edf0f2988 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 09:19:30.920696 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2259d1-8195-4241-bcac-0361733090e3 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:30.920989 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2259d1-8195-4241-bcac-0361733090e3 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:30.921269 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2259d1-8195-4241-bcac-0361733090e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:30.967176 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5393d69d-f9a0-45f4-b92d-afc78ff3d659 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:19:30.969151 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-5393d69d-f9a0-45f4-b92d-afc78ff3d659 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:19:30.971272 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-df2259d1-8195-4241-bcac-0361733090e3 None None] MFA Rules not processed for user `265fa1d06bc049db92316c145970f887`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:19:30.972645 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-df2259d1-8195-4241-bcac-0361733090e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:19:31.051087 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:19:31.066295 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5393d69d-f9a0-45f4-b92d-afc78ff3d659 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:19:31.066850 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1083/2186] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:19:30 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:19:31.072926 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a514f0-726e-4439-8578-9fa7c091f2c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:19:31.073385 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a514f0-726e-4439-8578-9fa7c091f2c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:31.073667 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a514f0-726e-4439-8578-9fa7c091f2c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:31.074583 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a514f0-726e-4439-8578-9fa7c091f2c6 admin admin] PATH_INFO: `/v3/projects/7a5344011639483e98b89356e40d8d81` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:31.075929 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a514f0-726e-4439-8578-9fa7c091f2c6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7a5344011639483e98b89356e40d8d81)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:19:31.078246 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a514f0-726e-4439-8578-9fa7c091f2c6 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:19:31.084423 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2259d1-8195-4241-bcac-0361733090e3 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:19:31.086885 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-df2259d1-8195-4241-bcac-0361733090e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:19:31.143700 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-d6a514f0-726e-4439-8578-9fa7c091f2c6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a5344011639483e98b89356e40d8d81', 'request_id': 'req-d6a514f0-726e-4439-8578-9fa7c091f2c6'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:19:31.151812 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-d6a514f0-726e-4439-8578-9fa7c091f2c6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a5344011639483e98b89356e40d8d81', 'request_id': 'req-d6a514f0-726e-4439-8578-9fa7c091f2c6'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:19:31.170070 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-d6a514f0-726e-4439-8578-9fa7c091f2c6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a5344011639483e98b89356e40d8d81', 'request_id': 'req-d6a514f0-726e-4439-8578-9fa7c091f2c6'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:19:31.179441 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2259d1-8195-4241-bcac-0361733090e3 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:19:31.191377 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-d6a514f0-726e-4439-8578-9fa7c091f2c6 admin admin] The token cache is being invalidate because project 7a5344011639483e98b89356e40d8d81 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80382) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 09:19:31.192152 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-d6a514f0-726e-4439-8578-9fa7c091f2c6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:19:31.192919 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-d6a514f0-726e-4439-8578-9fa7c091f2c6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:19:31.195228 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1104/2187] 10.222.0.26 () {58 vars in 1232 bytes} [Thu Jul 14 09:19:30 2022] DELETE /identity/v3/projects/7a5344011639483e98b89356e40d8d81 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:19:31.200329 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423d9aaa-9fef-4923-a494-ab7b5e8dbdcb None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:31.201282 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423d9aaa-9fef-4923-a494-ab7b5e8dbdcb None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:31.201952 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423d9aaa-9fef-4923-a494-ab7b5e8dbdcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:31.222963 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-423d9aaa-9fef-4923-a494-ab7b5e8dbdcb None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:19:31.224213 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-423d9aaa-9fef-4923-a494-ab7b5e8dbdcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:19:31.266158 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cf04f2-e2dd-4d54-94a9-380b43aec0d3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e76f1952a5b146808844b9f2b7395035', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:19:31.267098 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cf04f2-e2dd-4d54-94a9-380b43aec0d3 service cinder] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:31.267452 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cf04f2-e2dd-4d54-94a9-380b43aec0d3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:31.267734 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cf04f2-e2dd-4d54-94a9-380b43aec0d3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:31.277862 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-47cf04f2-e2dd-4d54-94a9-380b43aec0d3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:19:31.332391 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1105/2188] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:19:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 09:19:31.340650 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8a2cbb-2131-48d6-9969-f25cb369739b None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:31.341048 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8a2cbb-2131-48d6-9969-f25cb369739b None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:31.341425 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8a2cbb-2131-48d6-9969-f25cb369739b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:31.364551 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-ea8a2cbb-2131-48d6-9969-f25cb369739b None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:19:31.366403 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-ea8a2cbb-2131-48d6-9969-f25cb369739b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:19:31.376492 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1106/2189] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:19:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 09:19:31.381311 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8a2cbb-2131-48d6-9969-f25cb369739b None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:19:31.392959 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8a2cbb-2131-48d6-9969-f25cb369739b None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:19:31.398304 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:19:31.400864 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18182e3a-8cbf-4a51-8bee-1faf4a594a06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:19:31.401657 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18182e3a-8cbf-4a51-8bee-1faf4a594a06 admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:31.401876 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18182e3a-8cbf-4a51-8bee-1faf4a594a06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:31.402204 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18182e3a-8cbf-4a51-8bee-1faf4a594a06 admin admin] PATH_INFO: `/v3/domains` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:31.403277 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18182e3a-8cbf-4a51-8bee-1faf4a594a06 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:19:31.403618 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18182e3a-8cbf-4a51-8bee-1faf4a594a06 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:19:31.406322 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18182e3a-8cbf-4a51-8bee-1faf4a594a06 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:19:31.418851 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cf04f2-e2dd-4d54-94a9-380b43aec0d3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:19:31.421235 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cf04f2-e2dd-4d54-94a9-380b43aec0d3 service cinder] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:19:31.435438 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1084/2190] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:19:31 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 09:19:31.437601 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cf04f2-e2dd-4d54-94a9-380b43aec0d3 service cinder] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:19:31.438919 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1107/2191] 10.222.0.26 () {58 vars in 1152 bytes} [Thu Jul 14 09:19:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 09:19:31.453671 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cf04f2-e2dd-4d54-94a9-380b43aec0d3 service cinder] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:19:31.461161 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677c25e6-3310-4eae-bbeb-7107b0fc7b1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:19:31.461933 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677c25e6-3310-4eae-bbeb-7107b0fc7b1c admin admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:31.462225 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677c25e6-3310-4eae-bbeb-7107b0fc7b1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:31.462554 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677c25e6-3310-4eae-bbeb-7107b0fc7b1c admin admin] PATH_INFO: `/v3/projects` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:31.463422 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677c25e6-3310-4eae-bbeb-7107b0fc7b1c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:19:31.465339 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677c25e6-3310-4eae-bbeb-7107b0fc7b1c admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:19:31.476345 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18182e3a-8cbf-4a51-8bee-1faf4a594a06 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:19:31.478996 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-18182e3a-8cbf-4a51-8bee-1faf4a594a06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:19:31.497522 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18182e3a-8cbf-4a51-8bee-1faf4a594a06 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:19:31.511022 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac494cde-3815-4625-b86f-71b473185c28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:19:31.511316 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1085/2192] 10.222.0.26 () {60 vars in 1152 bytes} [Thu Jul 14 09:19:31 2022] POST /identity/v3/projects => generated 399 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 09:19:31.512686 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac494cde-3815-4625-b86f-71b473185c28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:31.512965 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac494cde-3815-4625-b86f-71b473185c28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:31.513241 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac494cde-3815-4625-b86f-71b473185c28 admin admin] PATH_INFO: `/v3/users/265fa1d06bc049db92316c145970f887` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:31.515950 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac494cde-3815-4625-b86f-71b473185c28 admin admin] RBAC: Authorizing `identity:delete_user(user_id=265fa1d06bc049db92316c145970f887)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:19:31.517146 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677c25e6-3310-4eae-bbeb-7107b0fc7b1c admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:19:31.519360 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac494cde-3815-4625-b86f-71b473185c28 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:19:31.630719 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-ac494cde-3815-4625-b86f-71b473185c28 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '265fa1d06bc049db92316c145970f887'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:19:31.634743 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-ac494cde-3815-4625-b86f-71b473185c28 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '265fa1d06bc049db92316c145970f887'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:19:31.656177 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677c25e6-3310-4eae-bbeb-7107b0fc7b1c admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:19:31.657293 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:19:31.663147 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16788557-9770-4db6-b567-9dd16fd27a5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:19:31.664295 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16788557-9770-4db6-b567-9dd16fd27a5a admin admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:31.664545 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-ac494cde-3815-4625-b86f-71b473185c28 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '265fa1d06bc049db92316c145970f887'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:19:31.665174 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16788557-9770-4db6-b567-9dd16fd27a5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:31.665997 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16788557-9770-4db6-b567-9dd16fd27a5a admin admin] PATH_INFO: `/v3/users` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:31.666858 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16788557-9770-4db6-b567-9dd16fd27a5a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:19:31.669570 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16788557-9770-4db6-b567-9dd16fd27a5a admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:19:31.677499 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-ac494cde-3815-4625-b86f-71b473185c28 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '265fa1d06bc049db92316c145970f887'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:19:31.678478 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-ac494cde-3815-4625-b86f-71b473185c28 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '265fa1d06bc049db92316c145970f887'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:19:31.696715 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1108/2193] 10.222.0.26 () {58 vars in 1223 bytes} [Thu Jul 14 09:19:31 2022] DELETE /identity/v3/users/265fa1d06bc049db92316c145970f887 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:19:31.698170 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345d682f-7724-42bc-9314-3b186d30b234 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:31.698462 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345d682f-7724-42bc-9314-3b186d30b234 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:31.699082 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345d682f-7724-42bc-9314-3b186d30b234 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:31.706330 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1086/2194] 10.222.0.26 () {60 vars in 1143 bytes} [Thu Jul 14 09:19:31 2022] POST /identity/v3/users => generated 300 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 09:19:31.710824 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16788557-9770-4db6-b567-9dd16fd27a5a admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:19:31.715885 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-16788557-9770-4db6-b567-9dd16fd27a5a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:19:31.717997 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-345d682f-7724-42bc-9314-3b186d30b234 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:19:31.719519 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-345d682f-7724-42bc-9314-3b186d30b234 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:19:31.749242 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16788557-9770-4db6-b567-9dd16fd27a5a admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:19:31.795486 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1109/2195] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:19:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:19:31.845649 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345d682f-7724-42bc-9314-3b186d30b234 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:19:31.847727 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-345d682f-7724-42bc-9314-3b186d30b234 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:19:31.861003 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f551ebb9-04f5-4f3e-9b34-0b4f50d9191b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:19:31.861931 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f551ebb9-04f5-4f3e-9b34-0b4f50d9191b admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:31.862297 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f551ebb9-04f5-4f3e-9b34-0b4f50d9191b admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:31.862639 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f551ebb9-04f5-4f3e-9b34-0b4f50d9191b admin admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:31.863654 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f551ebb9-04f5-4f3e-9b34-0b4f50d9191b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:19:31.865471 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f551ebb9-04f5-4f3e-9b34-0b4f50d9191b admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:19:31.884793 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1087/2196] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jul 14 09:19:31 2022] GET /identity/v3/roles => generated 1420 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 09:19:31.885953 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f551ebb9-04f5-4f3e-9b34-0b4f50d9191b admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:19:31.896752 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f551ebb9-04f5-4f3e-9b34-0b4f50d9191b admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:19:31.903325 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06226ae9-b4be-4438-9f94-b6774352e72e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:19:31.913382 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06226ae9-b4be-4438-9f94-b6774352e72e admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:31.913765 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06226ae9-b4be-4438-9f94-b6774352e72e admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:31.914278 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06226ae9-b4be-4438-9f94-b6774352e72e admin admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:31.915182 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06226ae9-b4be-4438-9f94-b6774352e72e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:19:31.916978 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06226ae9-b4be-4438-9f94-b6774352e72e admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:19:31.937454 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1088/2197] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jul 14 09:19:31 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 09:19:31.941133 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06226ae9-b4be-4438-9f94-b6774352e72e admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:19:31.956860 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06226ae9-b4be-4438-9f94-b6774352e72e admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:19:31.963759 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e72df72-0fbc-440c-956c-80ddc57603b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:19:31.965204 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e72df72-0fbc-440c-956c-80ddc57603b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:31.965702 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e72df72-0fbc-440c-956c-80ddc57603b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:31.966286 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e72df72-0fbc-440c-956c-80ddc57603b9 admin admin] PATH_INFO: `/v3/projects/43bc8307b0d04e3e9285b68d016c70ff/users/c9d236a9974f4ab89717e78b5a41a8ab/roles/d1691322ce6a4edab7ce722edf0f2988` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:31.980864 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345d682f-7724-42bc-9314-3b186d30b234 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:19:31.985400 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118d40f6-4cc3-42a8-a4c6-5913709a4f04 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '952911c492444ffea4b211a24b564117', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:19:31.989049 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118d40f6-4cc3-42a8-a4c6-5913709a4f04 service neutron] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:31.989049 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118d40f6-4cc3-42a8-a4c6-5913709a4f04 service neutron] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:31.989389 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118d40f6-4cc3-42a8-a4c6-5913709a4f04 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:31.997508 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e72df72-0fbc-440c-956c-80ddc57603b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=43bc8307b0d04e3e9285b68d016c70ff, user_id=c9d236a9974f4ab89717e78b5a41a8ab, role_id=d1691322ce6a4edab7ce722edf0f2988)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:19:32.004867 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118d40f6-4cc3-42a8-a4c6-5913709a4f04 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:19:32.006470 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118d40f6-4cc3-42a8-a4c6-5913709a4f04 service neutron] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:19:32.006667 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e72df72-0fbc-440c-956c-80ddc57603b9 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:19:32.027392 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1110/2198] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:19:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 09:19:32.030065 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1089/2199] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jul 14 09:19:31 2022] PUT /identity/v3/projects/43bc8307b0d04e3e9285b68d016c70ff/users/c9d236a9974f4ab89717e78b5a41a8ab/roles/d1691322ce6a4edab7ce722edf0f2988 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:19:32.046210 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5785caa-8f8b-4052-bc03-48b26b662284 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:32.046503 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5785caa-8f8b-4052-bc03-48b26b662284 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:32.047146 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5785caa-8f8b-4052-bc03-48b26b662284 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:32.095745 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-a5785caa-8f8b-4052-bc03-48b26b662284 None None] MFA Rules not processed for user `c9d236a9974f4ab89717e78b5a41a8ab`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:19:32.097084 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-a5785caa-8f8b-4052-bc03-48b26b662284 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:19:32.103241 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:19:32.216140 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1090/2200] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:19:32 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:19:32.229632 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118d40f6-4cc3-42a8-a4c6-5913709a4f04 service neutron] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:19:32.231136 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-118d40f6-4cc3-42a8-a4c6-5913709a4f04 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:19:32.335912 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118d40f6-4cc3-42a8-a4c6-5913709a4f04 service neutron] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:19:32.424089 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:19:32.454311 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa5b742-60c0-4847-b9ba-0e5b0927ddca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e76f1952a5b146808844b9f2b7395035', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:19:32.454994 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa5b742-60c0-4847-b9ba-0e5b0927ddca service cinder] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:32.455310 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa5b742-60c0-4847-b9ba-0e5b0927ddca service cinder] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:32.455582 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa5b742-60c0-4847-b9ba-0e5b0927ddca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:32.466590 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa5b742-60c0-4847-b9ba-0e5b0927ddca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:19:32.468363 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa5b742-60c0-4847-b9ba-0e5b0927ddca service cinder] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:19:32.480125 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1111/2201] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:19:32 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 251 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:19:32.737884 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5785caa-8f8b-4052-bc03-48b26b662284 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:19:32.739428 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-a5785caa-8f8b-4052-bc03-48b26b662284 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:19:32.834105 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5785caa-8f8b-4052-bc03-48b26b662284 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:19:32.840228 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0a74a9-4f1c-4bca-8ac9-184b88695288 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:19:32.841539 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0a74a9-4f1c-4bca-8ac9-184b88695288 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:32.841539 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0a74a9-4f1c-4bca-8ac9-184b88695288 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:32.841854 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0a74a9-4f1c-4bca-8ac9-184b88695288 admin admin] PATH_INFO: `/v3/projects/05d00975363a48f7b9d3efc4aa176f85` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:32.844094 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0a74a9-4f1c-4bca-8ac9-184b88695288 admin admin] RBAC: Authorizing `identity:delete_project(project_id=05d00975363a48f7b9d3efc4aa176f85)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:19:32.846264 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0a74a9-4f1c-4bca-8ac9-184b88695288 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:19:32.967376 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-de0a74a9-4f1c-4bca-8ac9-184b88695288 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '05d00975363a48f7b9d3efc4aa176f85', 'request_id': 'req-de0a74a9-4f1c-4bca-8ac9-184b88695288'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:19:32.989259 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-de0a74a9-4f1c-4bca-8ac9-184b88695288 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '05d00975363a48f7b9d3efc4aa176f85', 'request_id': 'req-de0a74a9-4f1c-4bca-8ac9-184b88695288'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:19:33.018764 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-de0a74a9-4f1c-4bca-8ac9-184b88695288 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '05d00975363a48f7b9d3efc4aa176f85', 'request_id': 'req-de0a74a9-4f1c-4bca-8ac9-184b88695288'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:19:33.073374 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-de0a74a9-4f1c-4bca-8ac9-184b88695288 admin admin] The token cache is being invalidate because project 05d00975363a48f7b9d3efc4aa176f85 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80381) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 09:19:33.073877 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-de0a74a9-4f1c-4bca-8ac9-184b88695288 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:19:33.075302 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-de0a74a9-4f1c-4bca-8ac9-184b88695288 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:19:33.081538 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1091/2202] 10.222.0.26 () {58 vars in 1232 bytes} [Thu Jul 14 09:19:32 2022] DELETE /identity/v3/projects/05d00975363a48f7b9d3efc4aa176f85 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:19:33.407524 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52e67c5-e1a7-4631-926b-beaccb138add None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:33.408061 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52e67c5-e1a7-4631-926b-beaccb138add None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:33.408168 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52e67c5-e1a7-4631-926b-beaccb138add None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:33.436124 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-f52e67c5-e1a7-4631-926b-beaccb138add None None] MFA Rules not processed for user `e072ca213ed74cbda810a1fd971ae23d`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:19:33.437333 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-f52e67c5-e1a7-4631-926b-beaccb138add None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:19:33.516202 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1112/2203] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:19:33 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:19:33.520723 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0a74a9-4f1c-4bca-8ac9-184b88695288 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:19:33.522114 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-de0a74a9-4f1c-4bca-8ac9-184b88695288 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:19:33.567298 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:19:33.592318 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0a74a9-4f1c-4bca-8ac9-184b88695288 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:19:33.597040 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f711ac41-bacc-4bee-8b91-46ae581d8100 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:19:33.597696 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f711ac41-bacc-4bee-8b91-46ae581d8100 admin admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:33.597946 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f711ac41-bacc-4bee-8b91-46ae581d8100 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:33.598216 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f711ac41-bacc-4bee-8b91-46ae581d8100 admin admin] PATH_INFO: `/v3/projects` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:33.598904 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f711ac41-bacc-4bee-8b91-46ae581d8100 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:19:33.600832 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f711ac41-bacc-4bee-8b91-46ae581d8100 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:19:33.627660 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1092/2204] 10.222.0.26 () {60 vars in 1152 bytes} [Thu Jul 14 09:19:33 2022] POST /identity/v3/projects => generated 379 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 09:19:33.631955 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52e67c5-e1a7-4631-926b-beaccb138add None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:19:33.642878 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52e67c5-e1a7-4631-926b-beaccb138add None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:19:33.705017 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc36b30b-0be1-4531-96f1-b6953db68475 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:19:33.705693 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc36b30b-0be1-4531-96f1-b6953db68475 admin admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:33.705977 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc36b30b-0be1-4531-96f1-b6953db68475 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:33.706253 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc36b30b-0be1-4531-96f1-b6953db68475 admin admin] PATH_INFO: `/v3/users` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:33.706943 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc36b30b-0be1-4531-96f1-b6953db68475 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:19:33.708924 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc36b30b-0be1-4531-96f1-b6953db68475 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:19:33.729511 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1113/2205] 10.222.0.26 () {60 vars in 1143 bytes} [Thu Jul 14 09:19:33 2022] POST /identity/v3/users => generated 296 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 09:19:33.733733 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f711ac41-bacc-4bee-8b91-46ae581d8100 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:19:33.747032 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f711ac41-bacc-4bee-8b91-46ae581d8100 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:19:33.753444 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9681ae84-dae0-437c-aece-b0b07220089d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:19:33.754112 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9681ae84-dae0-437c-aece-b0b07220089d admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:33.754403 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9681ae84-dae0-437c-aece-b0b07220089d admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:33.754680 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9681ae84-dae0-437c-aece-b0b07220089d admin admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:33.755529 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9681ae84-dae0-437c-aece-b0b07220089d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:19:33.757145 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9681ae84-dae0-437c-aece-b0b07220089d admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:19:33.768876 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1093/2206] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jul 14 09:19:33 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 09:19:33.773689 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc36b30b-0be1-4531-96f1-b6953db68475 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:19:33.785186 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc36b30b-0be1-4531-96f1-b6953db68475 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:19:33.790900 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0226718-417b-45b6-bf4c-ec996698c109 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:19:33.791839 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0226718-417b-45b6-bf4c-ec996698c109 admin admin] REQUEST_METHOD: `PUT` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:33.792116 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0226718-417b-45b6-bf4c-ec996698c109 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:33.792387 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0226718-417b-45b6-bf4c-ec996698c109 admin admin] PATH_INFO: `/v3/projects/8f1ae6eac2dd48cba8bf16a163d69ee4/users/cdb088b3e09f4a80a58b9b12b164c48a/roles/7b549d58d1724395b343609d895c80d0` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:33.811077 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0226718-417b-45b6-bf4c-ec996698c109 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f1ae6eac2dd48cba8bf16a163d69ee4, user_id=cdb088b3e09f4a80a58b9b12b164c48a, role_id=7b549d58d1724395b343609d895c80d0)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:19:33.813795 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0226718-417b-45b6-bf4c-ec996698c109 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:19:33.829389 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1114/2207] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jul 14 09:19:33 2022] PUT /identity/v3/projects/8f1ae6eac2dd48cba8bf16a163d69ee4/users/cdb088b3e09f4a80a58b9b12b164c48a/roles/7b549d58d1724395b343609d895c80d0 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:19:33.833391 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9681ae84-dae0-437c-aece-b0b07220089d admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:19:33.843826 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9681ae84-dae0-437c-aece-b0b07220089d admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:19:33.902478 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68b2d15-f978-464a-9fd8-f52771c7b10b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:19:33.903490 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68b2d15-f978-464a-9fd8-f52771c7b10b admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:33.903490 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68b2d15-f978-464a-9fd8-f52771c7b10b admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:33.903761 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68b2d15-f978-464a-9fd8-f52771c7b10b admin admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:33.904620 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68b2d15-f978-464a-9fd8-f52771c7b10b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:19:33.906320 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68b2d15-f978-464a-9fd8-f52771c7b10b admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:19:33.917528 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1094/2208] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jul 14 09:19:33 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 09:19:33.921856 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0226718-417b-45b6-bf4c-ec996698c109 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:19:33.934553 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0226718-417b-45b6-bf4c-ec996698c109 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:19:33.940121 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82866825-035e-4f9c-9a71-5caa1ee9cfdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:19:33.941027 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82866825-035e-4f9c-9a71-5caa1ee9cfdc admin admin] REQUEST_METHOD: `PUT` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:33.941427 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82866825-035e-4f9c-9a71-5caa1ee9cfdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:33.941716 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82866825-035e-4f9c-9a71-5caa1ee9cfdc admin admin] PATH_INFO: `/v3/projects/8f1ae6eac2dd48cba8bf16a163d69ee4/users/cdb088b3e09f4a80a58b9b12b164c48a/roles/7b549d58d1724395b343609d895c80d0` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:33.943849 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82866825-035e-4f9c-9a71-5caa1ee9cfdc admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f1ae6eac2dd48cba8bf16a163d69ee4, user_id=cdb088b3e09f4a80a58b9b12b164c48a, role_id=7b549d58d1724395b343609d895c80d0)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:19:33.946522 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82866825-035e-4f9c-9a71-5caa1ee9cfdc admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:19:33.958029 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1115/2209] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jul 14 09:19:33 2022] PUT /identity/v3/projects/8f1ae6eac2dd48cba8bf16a163d69ee4/users/cdb088b3e09f4a80a58b9b12b164c48a/roles/7b549d58d1724395b343609d895c80d0 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:19:39.208262 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35843db3-7352-4f50-b514-b1ee826cee80 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:39.208802 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35843db3-7352-4f50-b514-b1ee826cee80 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:39.208992 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35843db3-7352-4f50-b514-b1ee826cee80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:39.253777 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-35843db3-7352-4f50-b514-b1ee826cee80 None None] MFA Rules not processed for user `cdb088b3e09f4a80a58b9b12b164c48a`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:19:39.255237 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-35843db3-7352-4f50-b514-b1ee826cee80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:19:39.392287 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1095/2210] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:19:39 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:19:39.409169 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82866825-035e-4f9c-9a71-5caa1ee9cfdc admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:19:39.410953 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-82866825-035e-4f9c-9a71-5caa1ee9cfdc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:19:39.439499 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:19:39.521818 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82866825-035e-4f9c-9a71-5caa1ee9cfdc admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:19:39.527796 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fff0838-4edc-41c9-885c-1b85ceeb2e22 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '538fff3c5a2b487eb44ae7c1a1198102', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:19:39.528465 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fff0838-4edc-41c9-885c-1b85ceeb2e22 service nova] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:39.528749 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fff0838-4edc-41c9-885c-1b85ceeb2e22 service nova] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:39.529028 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fff0838-4edc-41c9-885c-1b85ceeb2e22 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:39.595667 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fff0838-4edc-41c9-885c-1b85ceeb2e22 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:19:39.597624 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fff0838-4edc-41c9-885c-1b85ceeb2e22 service nova] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:19:39.607984 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1116/2211] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:19:39 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:19:46.537003 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35843db3-7352-4f50-b514-b1ee826cee80 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:19:46.538522 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-35843db3-7352-4f50-b514-b1ee826cee80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:19:46.577512 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:19:46.609498 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35843db3-7352-4f50-b514-b1ee826cee80 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:19:46.613990 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d934fa-a8ba-4a3a-aa38-eda3dc4c57fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:19:46.614954 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d934fa-a8ba-4a3a-aa38-eda3dc4c57fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:46.615541 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d934fa-a8ba-4a3a-aa38-eda3dc4c57fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:46.615859 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d934fa-a8ba-4a3a-aa38-eda3dc4c57fd admin admin] PATH_INFO: `/v3/users/c9d236a9974f4ab89717e78b5a41a8ab` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:46.617051 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d934fa-a8ba-4a3a-aa38-eda3dc4c57fd admin admin] RBAC: Authorizing `identity:delete_user(user_id=c9d236a9974f4ab89717e78b5a41a8ab)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:19:46.619027 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d934fa-a8ba-4a3a-aa38-eda3dc4c57fd admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:19:46.670380 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-84d934fa-a8ba-4a3a-aa38-eda3dc4c57fd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c9d236a9974f4ab89717e78b5a41a8ab'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:19:46.678432 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-84d934fa-a8ba-4a3a-aa38-eda3dc4c57fd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c9d236a9974f4ab89717e78b5a41a8ab'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:19:46.679089 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-84d934fa-a8ba-4a3a-aa38-eda3dc4c57fd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c9d236a9974f4ab89717e78b5a41a8ab'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:19:46.679837 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-84d934fa-a8ba-4a3a-aa38-eda3dc4c57fd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c9d236a9974f4ab89717e78b5a41a8ab'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:19:46.686866 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-84d934fa-a8ba-4a3a-aa38-eda3dc4c57fd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c9d236a9974f4ab89717e78b5a41a8ab'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:19:46.706198 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1096/2212] 10.222.0.26 () {58 vars in 1223 bytes} [Thu Jul 14 09:19:46 2022] DELETE /identity/v3/users/c9d236a9974f4ab89717e78b5a41a8ab => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:19:46.710472 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690e0f74-6e3e-463d-b479-4fe0c3dceaa0 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:46.710932 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690e0f74-6e3e-463d-b479-4fe0c3dceaa0 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:46.711025 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690e0f74-6e3e-463d-b479-4fe0c3dceaa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:46.728964 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-690e0f74-6e3e-463d-b479-4fe0c3dceaa0 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:19:46.730165 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-690e0f74-6e3e-463d-b479-4fe0c3dceaa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:19:46.798154 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1117/2213] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:19:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:19:46.808565 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d934fa-a8ba-4a3a-aa38-eda3dc4c57fd admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:19:46.810188 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-84d934fa-a8ba-4a3a-aa38-eda3dc4c57fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:19:46.880930 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d934fa-a8ba-4a3a-aa38-eda3dc4c57fd admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:19:46.885181 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b79e1d-bca2-439d-8365-4493b2d06e9f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '952911c492444ffea4b211a24b564117', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:19:46.885840 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b79e1d-bca2-439d-8365-4493b2d06e9f service neutron] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:46.886119 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b79e1d-bca2-439d-8365-4493b2d06e9f service neutron] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:46.886549 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b79e1d-bca2-439d-8365-4493b2d06e9f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:46.896927 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b79e1d-bca2-439d-8365-4493b2d06e9f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:19:46.898353 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b79e1d-bca2-439d-8365-4493b2d06e9f service neutron] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:19:46.908962 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1097/2214] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:19:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 09:19:47.223668 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690e0f74-6e3e-463d-b479-4fe0c3dceaa0 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:19:47.225543 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-690e0f74-6e3e-463d-b479-4fe0c3dceaa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:19:47.239621 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690e0f74-6e3e-463d-b479-4fe0c3dceaa0 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:19:47.272317 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e95851-c966-46a9-a0cb-462dedfc73c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:19:47.273221 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e95851-c966-46a9-a0cb-462dedfc73c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:47.273409 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e95851-c966-46a9-a0cb-462dedfc73c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:47.273988 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e95851-c966-46a9-a0cb-462dedfc73c9 admin admin] PATH_INFO: `/v3/projects/43bc8307b0d04e3e9285b68d016c70ff` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:47.275668 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e95851-c966-46a9-a0cb-462dedfc73c9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=43bc8307b0d04e3e9285b68d016c70ff)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:19:47.278201 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e95851-c966-46a9-a0cb-462dedfc73c9 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:19:47.356187 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-73e95851-c966-46a9-a0cb-462dedfc73c9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '43bc8307b0d04e3e9285b68d016c70ff', 'request_id': 'req-73e95851-c966-46a9-a0cb-462dedfc73c9'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:19:47.359256 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:19:47.364770 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-73e95851-c966-46a9-a0cb-462dedfc73c9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '43bc8307b0d04e3e9285b68d016c70ff', 'request_id': 'req-73e95851-c966-46a9-a0cb-462dedfc73c9'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:19:47.373042 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-73e95851-c966-46a9-a0cb-462dedfc73c9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '43bc8307b0d04e3e9285b68d016c70ff', 'request_id': 'req-73e95851-c966-46a9-a0cb-462dedfc73c9'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:19:47.390440 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-73e95851-c966-46a9-a0cb-462dedfc73c9 admin admin] The token cache is being invalidate because project 43bc8307b0d04e3e9285b68d016c70ff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80382) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 09:19:47.390866 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-73e95851-c966-46a9-a0cb-462dedfc73c9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:19:47.391887 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-73e95851-c966-46a9-a0cb-462dedfc73c9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:19:47.393789 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1118/2215] 10.222.0.26 () {58 vars in 1232 bytes} [Thu Jul 14 09:19:47 2022] DELETE /identity/v3/projects/43bc8307b0d04e3e9285b68d016c70ff => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:19:52.064201 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b79e1d-bca2-439d-8365-4493b2d06e9f service neutron] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:19:52.065905 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-90b79e1d-bca2-439d-8365-4493b2d06e9f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:19:52.079801 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b79e1d-bca2-439d-8365-4493b2d06e9f service neutron] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:19:52.146350 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:19:52.154590 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44dd3b3b-e340-499b-9601-7707601b40cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:19:52.155635 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dd3b3b-e340-499b-9601-7707601b40cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:52.155974 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dd3b3b-e340-499b-9601-7707601b40cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:52.156286 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dd3b3b-e340-499b-9601-7707601b40cb admin admin] PATH_INFO: `/v3/users/8baac06bf33a4f2182a30917293666ac` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:52.157809 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44dd3b3b-e340-499b-9601-7707601b40cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=8baac06bf33a4f2182a30917293666ac)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:19:52.160045 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44dd3b3b-e340-499b-9601-7707601b40cb admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:19:52.207715 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-44dd3b3b-e340-499b-9601-7707601b40cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8baac06bf33a4f2182a30917293666ac'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:19:52.217298 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-44dd3b3b-e340-499b-9601-7707601b40cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8baac06bf33a4f2182a30917293666ac'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:19:52.218190 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-44dd3b3b-e340-499b-9601-7707601b40cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8baac06bf33a4f2182a30917293666ac'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:19:52.218979 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-44dd3b3b-e340-499b-9601-7707601b40cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8baac06bf33a4f2182a30917293666ac'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:19:52.227289 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-44dd3b3b-e340-499b-9601-7707601b40cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8baac06bf33a4f2182a30917293666ac'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:19:52.244885 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1098/2216] 10.222.0.26 () {58 vars in 1223 bytes} [Thu Jul 14 09:19:52 2022] DELETE /identity/v3/users/8baac06bf33a4f2182a30917293666ac => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:19:52.249594 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b98665e-78fc-4518-9c84-2b103ba652ca None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:52.250067 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b98665e-78fc-4518-9c84-2b103ba652ca None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:52.250162 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b98665e-78fc-4518-9c84-2b103ba652ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:52.268057 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-4b98665e-78fc-4518-9c84-2b103ba652ca None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:19:52.269267 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-4b98665e-78fc-4518-9c84-2b103ba652ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:19:52.332424 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1119/2217] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:19:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:19:52.345413 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44dd3b3b-e340-499b-9601-7707601b40cb admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:19:52.347099 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-44dd3b3b-e340-499b-9601-7707601b40cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:19:52.408906 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44dd3b3b-e340-499b-9601-7707601b40cb admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:19:52.412826 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982ac5a1-bfea-43fa-a36c-385924ee89e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '952911c492444ffea4b211a24b564117', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:19:52.413480 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982ac5a1-bfea-43fa-a36c-385924ee89e4 service neutron] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:52.413759 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982ac5a1-bfea-43fa-a36c-385924ee89e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:52.414038 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982ac5a1-bfea-43fa-a36c-385924ee89e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:52.428929 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982ac5a1-bfea-43fa-a36c-385924ee89e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:19:52.430648 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982ac5a1-bfea-43fa-a36c-385924ee89e4 service neutron] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:19:52.442075 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1099/2218] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:19:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:19:52.745333 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b98665e-78fc-4518-9c84-2b103ba652ca None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:19:52.747175 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-4b98665e-78fc-4518-9c84-2b103ba652ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:19:52.760193 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b98665e-78fc-4518-9c84-2b103ba652ca None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:19:52.765698 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d049fc-030a-4237-b4be-b900e0f8498c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:19:52.767219 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d049fc-030a-4237-b4be-b900e0f8498c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:52.767553 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d049fc-030a-4237-b4be-b900e0f8498c admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:52.767858 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d049fc-030a-4237-b4be-b900e0f8498c admin admin] PATH_INFO: `/v3/projects/2abd5167cfff4a66b972cda4b9fb8b4f` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:52.769413 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d049fc-030a-4237-b4be-b900e0f8498c admin admin] RBAC: Authorizing `identity:delete_project(project_id=2abd5167cfff4a66b972cda4b9fb8b4f)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:19:52.771433 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d049fc-030a-4237-b4be-b900e0f8498c admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:19:52.820934 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-64d049fc-030a-4237-b4be-b900e0f8498c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2abd5167cfff4a66b972cda4b9fb8b4f', 'request_id': 'req-64d049fc-030a-4237-b4be-b900e0f8498c'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:19:52.827888 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-64d049fc-030a-4237-b4be-b900e0f8498c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2abd5167cfff4a66b972cda4b9fb8b4f', 'request_id': 'req-64d049fc-030a-4237-b4be-b900e0f8498c'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:19:52.837601 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-64d049fc-030a-4237-b4be-b900e0f8498c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2abd5167cfff4a66b972cda4b9fb8b4f', 'request_id': 'req-64d049fc-030a-4237-b4be-b900e0f8498c'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:19:52.853187 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-64d049fc-030a-4237-b4be-b900e0f8498c admin admin] The token cache is being invalidate because project 2abd5167cfff4a66b972cda4b9fb8b4f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80382) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 09:19:52.853480 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-64d049fc-030a-4237-b4be-b900e0f8498c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:19:52.854160 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-64d049fc-030a-4237-b4be-b900e0f8498c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:19:52.855889 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1120/2219] 10.222.0.26 () {58 vars in 1232 bytes} [Thu Jul 14 09:19:52 2022] DELETE /identity/v3/projects/2abd5167cfff4a66b972cda4b9fb8b4f => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:19:52.862227 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e758cb8-b57b-4900-9ca0-fd08b1c30a83 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:52.862559 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e758cb8-b57b-4900-9ca0-fd08b1c30a83 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:52.862841 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e758cb8-b57b-4900-9ca0-fd08b1c30a83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:52.881380 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-4e758cb8-b57b-4900-9ca0-fd08b1c30a83 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:19:52.882603 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-4e758cb8-b57b-4900-9ca0-fd08b1c30a83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:19:52.952986 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1100/2220] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:19:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:19:52.964006 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e70281-d2c7-44ae-a1a7-7d2940847bdc None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:52.964308 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e70281-d2c7-44ae-a1a7-7d2940847bdc None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:52.964582 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e70281-d2c7-44ae-a1a7-7d2940847bdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:52.983605 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-b0e70281-d2c7-44ae-a1a7-7d2940847bdc None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:19:52.984800 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-b0e70281-d2c7-44ae-a1a7-7d2940847bdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:19:52.993633 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1121/2221] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:19:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:19:52.997821 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e758cb8-b57b-4900-9ca0-fd08b1c30a83 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:19:53.007792 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e758cb8-b57b-4900-9ca0-fd08b1c30a83 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:19:53.013289 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd2bff1-c634-4eba-ae03-642c7e065035 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:19:53.013931 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd2bff1-c634-4eba-ae03-642c7e065035 admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:53.014212 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd2bff1-c634-4eba-ae03-642c7e065035 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:53.014482 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd2bff1-c634-4eba-ae03-642c7e065035 admin admin] PATH_INFO: `/v3/domains` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:53.015178 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd2bff1-c634-4eba-ae03-642c7e065035 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:19:53.015537 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd2bff1-c634-4eba-ae03-642c7e065035 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:19:53.017157 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd2bff1-c634-4eba-ae03-642c7e065035 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:19:53.029317 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1101/2222] 10.222.0.26 () {58 vars in 1152 bytes} [Thu Jul 14 09:19:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 09:19:53.033916 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e70281-d2c7-44ae-a1a7-7d2940847bdc None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:19:53.044598 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e70281-d2c7-44ae-a1a7-7d2940847bdc None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:19:53.050403 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9890887-1ad2-4f7c-a37e-bbfdbff9c3f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:19:53.051058 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9890887-1ad2-4f7c-a37e-bbfdbff9c3f1 admin admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:53.051436 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9890887-1ad2-4f7c-a37e-bbfdbff9c3f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:53.051695 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9890887-1ad2-4f7c-a37e-bbfdbff9c3f1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:53.052384 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9890887-1ad2-4f7c-a37e-bbfdbff9c3f1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:19:53.054286 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9890887-1ad2-4f7c-a37e-bbfdbff9c3f1 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:19:53.080026 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1122/2223] 10.222.0.26 () {60 vars in 1152 bytes} [Thu Jul 14 09:19:53 2022] POST /identity/v3/projects => generated 385 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 09:19:53.084408 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd2bff1-c634-4eba-ae03-642c7e065035 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:19:53.091417 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:19:53.095107 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd2bff1-c634-4eba-ae03-642c7e065035 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:19:53.160435 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddbf29d-5ceb-4a61-8519-053e1ca42cbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:19:53.161118 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddbf29d-5ceb-4a61-8519-053e1ca42cbf admin admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:53.161407 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddbf29d-5ceb-4a61-8519-053e1ca42cbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:53.161725 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddbf29d-5ceb-4a61-8519-053e1ca42cbf admin admin] PATH_INFO: `/v3/users` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:53.162444 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ddbf29d-5ceb-4a61-8519-053e1ca42cbf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:19:53.164431 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ddbf29d-5ceb-4a61-8519-053e1ca42cbf admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:19:53.187556 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1102/2224] 10.222.0.26 () {60 vars in 1143 bytes} [Thu Jul 14 09:19:53 2022] POST /identity/v3/users => generated 293 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 09:19:53.191864 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9890887-1ad2-4f7c-a37e-bbfdbff9c3f1 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:19:53.202492 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9890887-1ad2-4f7c-a37e-bbfdbff9c3f1 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:19:53.207237 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220cff14-c594-4527-8820-d8cc47422c38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:19:53.207909 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220cff14-c594-4527-8820-d8cc47422c38 admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:53.208183 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220cff14-c594-4527-8820-d8cc47422c38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:53.208454 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220cff14-c594-4527-8820-d8cc47422c38 admin admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:53.209184 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220cff14-c594-4527-8820-d8cc47422c38 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:19:53.210783 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220cff14-c594-4527-8820-d8cc47422c38 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:19:53.221462 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1123/2225] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jul 14 09:19:53 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 09:19:53.225482 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddbf29d-5ceb-4a61-8519-053e1ca42cbf admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:19:53.240771 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddbf29d-5ceb-4a61-8519-053e1ca42cbf admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:19:53.245766 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763989ca-615d-4a00-be9d-96fab3a57eca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:19:53.246438 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763989ca-615d-4a00-be9d-96fab3a57eca admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:53.246787 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763989ca-615d-4a00-be9d-96fab3a57eca admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:53.246971 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763989ca-615d-4a00-be9d-96fab3a57eca admin admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:53.247742 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763989ca-615d-4a00-be9d-96fab3a57eca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:19:53.249320 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763989ca-615d-4a00-be9d-96fab3a57eca admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:19:53.259947 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1103/2226] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jul 14 09:19:53 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 09:19:53.264269 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220cff14-c594-4527-8820-d8cc47422c38 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:19:53.274609 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220cff14-c594-4527-8820-d8cc47422c38 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:19:53.279259 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706427f0-f604-4706-a8ff-cf2f5391692a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:19:53.280160 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706427f0-f604-4706-a8ff-cf2f5391692a admin admin] REQUEST_METHOD: `PUT` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:53.280436 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706427f0-f604-4706-a8ff-cf2f5391692a admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:53.280699 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706427f0-f604-4706-a8ff-cf2f5391692a admin admin] PATH_INFO: `/v3/projects/e6ebe73241894d12a03f7fde53207274/users/b4dcd1f4a61643f69b81e7edae238fd2/roles/d1691322ce6a4edab7ce722edf0f2988` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:53.299367 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706427f0-f604-4706-a8ff-cf2f5391692a admin admin] RBAC: Authorizing `identity:create_grant(project_id=e6ebe73241894d12a03f7fde53207274, user_id=b4dcd1f4a61643f69b81e7edae238fd2, role_id=d1691322ce6a4edab7ce722edf0f2988)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:19:53.302424 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706427f0-f604-4706-a8ff-cf2f5391692a admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:19:53.315161 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:19:53.321953 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1124/2227] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jul 14 09:19:53 2022] PUT /identity/v3/projects/e6ebe73241894d12a03f7fde53207274/users/b4dcd1f4a61643f69b81e7edae238fd2/roles/d1691322ce6a4edab7ce722edf0f2988 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 09:19:53.327374 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbed3247-76a3-4518-b6b4-a6c1ee70b720 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:53.327975 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbed3247-76a3-4518-b6b4-a6c1ee70b720 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:53.328370 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbed3247-76a3-4518-b6b4-a6c1ee70b720 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:53.347040 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-cbed3247-76a3-4518-b6b4-a6c1ee70b720 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:19:53.348207 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-cbed3247-76a3-4518-b6b4-a6c1ee70b720 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:19:53.417726 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1104/2228] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:19:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:19:53.427488 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706427f0-f604-4706-a8ff-cf2f5391692a admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:19:53.429417 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-706427f0-f604-4706-a8ff-cf2f5391692a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:19:53.487106 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706427f0-f604-4706-a8ff-cf2f5391692a admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:19:53.491018 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5214913-9525-49a8-b23e-918f9b2be137 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '952911c492444ffea4b211a24b564117', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:19:53.491763 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5214913-9525-49a8-b23e-918f9b2be137 service neutron] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:53.492058 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5214913-9525-49a8-b23e-918f9b2be137 service neutron] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:53.492333 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5214913-9525-49a8-b23e-918f9b2be137 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:53.503051 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5214913-9525-49a8-b23e-918f9b2be137 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:19:53.504609 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5214913-9525-49a8-b23e-918f9b2be137 service neutron] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:19:53.515617 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1125/2229] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:19:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:19:56.472499 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fff9d23-b994-4b82-ac83-40ee84b5c57a None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:56.473080 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fff9d23-b994-4b82-ac83-40ee84b5c57a None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:56.473165 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fff9d23-b994-4b82-ac83-40ee84b5c57a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:56.501014 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-0fff9d23-b994-4b82-ac83-40ee84b5c57a None None] MFA Rules not processed for user `a2ab766e96024b899a01c540d9bd1322`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:19:56.502254 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-0fff9d23-b994-4b82-ac83-40ee84b5c57a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:19:56.587248 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1105/2230] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:19:56 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:19:59.065296 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1116517b-e66f-41e6-b828-9db033c025c5 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:59.066313 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1116517b-e66f-41e6-b828-9db033c025c5 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:59.066809 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1116517b-e66f-41e6-b828-9db033c025c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:59.104882 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-1116517b-e66f-41e6-b828-9db033c025c5 None None] MFA Rules not processed for user `b4dcd1f4a61643f69b81e7edae238fd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:19:59.106493 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-1116517b-e66f-41e6-b828-9db033c025c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:19:59.177320 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1126/2231] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:19:59 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:19:59.181960 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fff9d23-b994-4b82-ac83-40ee84b5c57a None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:19:59.192798 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fff9d23-b994-4b82-ac83-40ee84b5c57a None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:19:59.197261 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2185d997-9793-4bbb-8444-f0342e4a7160 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:19:59.198281 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2185d997-9793-4bbb-8444-f0342e4a7160 admin admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:59.198573 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2185d997-9793-4bbb-8444-f0342e4a7160 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:59.198849 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2185d997-9793-4bbb-8444-f0342e4a7160 admin admin] PATH_INFO: `/v3/projects` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:59.199595 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2185d997-9793-4bbb-8444-f0342e4a7160 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:19:59.201496 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2185d997-9793-4bbb-8444-f0342e4a7160 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:19:59.232119 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1106/2232] 10.222.0.26 () {60 vars in 1152 bytes} [Thu Jul 14 09:19:59 2022] POST /identity/v3/projects => generated 383 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 09:19:59.236551 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1116517b-e66f-41e6-b828-9db033c025c5 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:19:59.251530 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1116517b-e66f-41e6-b828-9db033c025c5 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:19:59.322600 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb886bd4-9cef-452e-b279-0f6cb3877669 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:19:59.323411 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb886bd4-9cef-452e-b279-0f6cb3877669 admin admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:59.323695 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb886bd4-9cef-452e-b279-0f6cb3877669 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:59.323968 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb886bd4-9cef-452e-b279-0f6cb3877669 admin admin] PATH_INFO: `/v3/users` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:59.324807 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb886bd4-9cef-452e-b279-0f6cb3877669 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:19:59.326761 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb886bd4-9cef-452e-b279-0f6cb3877669 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:19:59.348862 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1127/2233] 10.222.0.26 () {60 vars in 1143 bytes} [Thu Jul 14 09:19:59 2022] POST /identity/v3/users => generated 298 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 09:19:59.352899 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2185d997-9793-4bbb-8444-f0342e4a7160 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:19:59.363385 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2185d997-9793-4bbb-8444-f0342e4a7160 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:19:59.367967 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12edc49d-63ce-4523-8847-988551d1c647 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:19:59.368650 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12edc49d-63ce-4523-8847-988551d1c647 admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:59.368932 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12edc49d-63ce-4523-8847-988551d1c647 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:59.369211 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12edc49d-63ce-4523-8847-988551d1c647 admin admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:59.369939 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12edc49d-63ce-4523-8847-988551d1c647 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:19:59.371573 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12edc49d-63ce-4523-8847-988551d1c647 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:19:59.372848 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:19:59.383565 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1107/2234] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jul 14 09:19:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 09:19:59.387766 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb886bd4-9cef-452e-b279-0f6cb3877669 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:19:59.398470 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb886bd4-9cef-452e-b279-0f6cb3877669 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:19:59.405550 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9725ffd5-0b09-4ef8-9100-ae18350975d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:19:59.406649 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9725ffd5-0b09-4ef8-9100-ae18350975d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:59.407018 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9725ffd5-0b09-4ef8-9100-ae18350975d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:59.407358 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9725ffd5-0b09-4ef8-9100-ae18350975d9 admin admin] PATH_INFO: `/v3/projects/9b56c67e6cac415eb089c52f9cc0bf9c/users/43c335fb11c04f839050207590c31e92/roles/7b549d58d1724395b343609d895c80d0` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:59.424314 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9725ffd5-0b09-4ef8-9100-ae18350975d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b56c67e6cac415eb089c52f9cc0bf9c, user_id=43c335fb11c04f839050207590c31e92, role_id=7b549d58d1724395b343609d895c80d0)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:19:59.426950 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9725ffd5-0b09-4ef8-9100-ae18350975d9 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:19:59.435416 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:19:59.441066 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1128/2235] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jul 14 09:19:59 2022] PUT /identity/v3/projects/9b56c67e6cac415eb089c52f9cc0bf9c/users/43c335fb11c04f839050207590c31e92/roles/7b549d58d1724395b343609d895c80d0 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:19:59.445372 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12edc49d-63ce-4523-8847-988551d1c647 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:19:59.456751 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12edc49d-63ce-4523-8847-988551d1c647 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:19:59.518562 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc550f2-f245-46d6-962d-e175cbfc9a59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:19:59.519286 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc550f2-f245-46d6-962d-e175cbfc9a59 admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:59.519574 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc550f2-f245-46d6-962d-e175cbfc9a59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:59.519852 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc550f2-f245-46d6-962d-e175cbfc9a59 admin admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:59.520617 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc550f2-f245-46d6-962d-e175cbfc9a59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:19:59.522222 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc550f2-f245-46d6-962d-e175cbfc9a59 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:19:59.534984 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1108/2236] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jul 14 09:19:59 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 09:19:59.537151 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9725ffd5-0b09-4ef8-9100-ae18350975d9 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:19:59.548369 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9725ffd5-0b09-4ef8-9100-ae18350975d9 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:19:59.554351 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70361bc3-aa56-4bc4-966b-5d78e79d2fdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:19:59.554860 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70361bc3-aa56-4bc4-966b-5d78e79d2fdd admin admin] REQUEST_METHOD: `PUT` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:19:59.555146 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70361bc3-aa56-4bc4-966b-5d78e79d2fdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:19:59.555456 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70361bc3-aa56-4bc4-966b-5d78e79d2fdd admin admin] PATH_INFO: `/v3/projects/9b56c67e6cac415eb089c52f9cc0bf9c/users/43c335fb11c04f839050207590c31e92/roles/7b549d58d1724395b343609d895c80d0` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:19:59.557861 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70361bc3-aa56-4bc4-966b-5d78e79d2fdd admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b56c67e6cac415eb089c52f9cc0bf9c, user_id=43c335fb11c04f839050207590c31e92, role_id=7b549d58d1724395b343609d895c80d0)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:19:59.560785 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70361bc3-aa56-4bc4-966b-5d78e79d2fdd admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:19:59.572861 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1129/2237] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jul 14 09:19:59 2022] PUT /identity/v3/projects/9b56c67e6cac415eb089c52f9cc0bf9c/users/43c335fb11c04f839050207590c31e92/roles/7b549d58d1724395b343609d895c80d0 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:20:02.708653 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c282ae02-d7d5-4cf9-9e58-22ebf6a2da29 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:20:02.709565 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c282ae02-d7d5-4cf9-9e58-22ebf6a2da29 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:20:02.710008 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c282ae02-d7d5-4cf9-9e58-22ebf6a2da29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:20:02.730901 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-c282ae02-d7d5-4cf9-9e58-22ebf6a2da29 None None] MFA Rules not processed for user `e072ca213ed74cbda810a1fd971ae23d`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:20:02.732217 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-c282ae02-d7d5-4cf9-9e58-22ebf6a2da29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:20:02.858867 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1109/2238] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:20:02 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:20:07.044736 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16be1427-dee2-444c-81c1-2e69e17bc809 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:20:07.045290 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16be1427-dee2-444c-81c1-2e69e17bc809 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:20:07.045576 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16be1427-dee2-444c-81c1-2e69e17bc809 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:20:07.072177 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-16be1427-dee2-444c-81c1-2e69e17bc809 None None] MFA Rules not processed for user `18377cf2b2d14cb0b4cbda6ec720ed28`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:20:07.074844 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-16be1427-dee2-444c-81c1-2e69e17bc809 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:20:07.163966 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1130/2239] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:20:07 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:20:07.178834 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb744601-3110-4b87-85b9-13650e6b85ac None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:20:07.179451 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb744601-3110-4b87-85b9-13650e6b85ac None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:20:07.179534 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb744601-3110-4b87-85b9-13650e6b85ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:20:07.220951 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-eb744601-3110-4b87-85b9-13650e6b85ac None None] MFA Rules not processed for user `43c335fb11c04f839050207590c31e92`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:20:07.222400 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-eb744601-3110-4b87-85b9-13650e6b85ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:20:07.307470 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1110/2240] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:20:07 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:20:07.381177 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16be1427-dee2-444c-81c1-2e69e17bc809 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:20:07.382622 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-16be1427-dee2-444c-81c1-2e69e17bc809 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:20:07.451757 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16be1427-dee2-444c-81c1-2e69e17bc809 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:20:07.455869 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1a5bf2-66b6-4f63-8a0b-ac1a7c8e1aa3 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '872dbb91c9e54f13b1a31fc8242126df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:20:07.456611 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1a5bf2-66b6-4f63-8a0b-ac1a7c8e1aa3 service glance] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:20:07.456914 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1a5bf2-66b6-4f63-8a0b-ac1a7c8e1aa3 service glance] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:20:07.457193 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1a5bf2-66b6-4f63-8a0b-ac1a7c8e1aa3 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:20:07.568883 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1a5bf2-66b6-4f63-8a0b-ac1a7c8e1aa3 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:20:07.570584 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1a5bf2-66b6-4f63-8a0b-ac1a7c8e1aa3 service glance] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:20:07.582073 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1131/2241] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:20:07 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:20:07.708506 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85438642-c936-4b80-992d-c25e2b200d05 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:20:07.708895 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85438642-c936-4b80-992d-c25e2b200d05 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:20:07.709738 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85438642-c936-4b80-992d-c25e2b200d05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:20:07.731502 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-85438642-c936-4b80-992d-c25e2b200d05 None None] MFA Rules not processed for user `7d6267ede8d64e2abe1f4058a9797fb7`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:20:07.733010 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-85438642-c936-4b80-992d-c25e2b200d05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:20:07.761827 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:20:07.818024 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1111/2242] 10.222.0.26 () {58 vars in 1009 bytes} [Thu Jul 14 09:20:07 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:20:07.830831 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1a5bf2-66b6-4f63-8a0b-ac1a7c8e1aa3 service glance] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:20:07.832465 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-2e1a5bf2-66b6-4f63-8a0b-ac1a7c8e1aa3 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:20:07.887420 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:20:07.917492 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1a5bf2-66b6-4f63-8a0b-ac1a7c8e1aa3 service glance] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:20:07.924373 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f52ae6-53a9-435a-9b04-8b5f68fcb908 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5a43c80514b4a13af03c7f8ebcf7a5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:20:07.925570 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f52ae6-53a9-435a-9b04-8b5f68fcb908 service swift] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:20:07.925570 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f52ae6-53a9-435a-9b04-8b5f68fcb908 service swift] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:20:07.925754 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f52ae6-53a9-435a-9b04-8b5f68fcb908 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:20:07.936373 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f52ae6-53a9-435a-9b04-8b5f68fcb908 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:20:07.937746 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f52ae6-53a9-435a-9b04-8b5f68fcb908 service swift] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:20:07.947589 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1132/2243] 10.222.0.26 () {60 vars in 1384 bytes} [Thu Jul 14 09:20:07 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 14 09:20:17.810468 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=15 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:20:17.944077 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=12 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:20:20.432682 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94835793-ab06-4763-b64f-764dacba8a29 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:20:20.433314 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94835793-ab06-4763-b64f-764dacba8a29 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:20:20.433515 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94835793-ab06-4763-b64f-764dacba8a29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:20:20.453866 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-94835793-ab06-4763-b64f-764dacba8a29 None None] MFA Rules not processed for user `b4dcd1f4a61643f69b81e7edae238fd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:20:20.455436 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-94835793-ab06-4763-b64f-764dacba8a29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:20:20.467244 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1112/2244] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:20:20 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:20:24.591170 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ba7dbc-d041-449c-86ad-2501a6cc783b None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:20:24.596602 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ba7dbc-d041-449c-86ad-2501a6cc783b None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:20:24.597743 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ba7dbc-d041-449c-86ad-2501a6cc783b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:20:24.648447 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-b7ba7dbc-d041-449c-86ad-2501a6cc783b None None] MFA Rules not processed for user `7d6267ede8d64e2abe1f4058a9797fb7`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:20:24.652433 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-b7ba7dbc-d041-449c-86ad-2501a6cc783b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:20:24.688275 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1133/2245] 10.222.0.26 () {58 vars in 1009 bytes} [Thu Jul 14 09:20:24 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:20:24.708182 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94835793-ab06-4763-b64f-764dacba8a29 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:20:24.725683 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94835793-ab06-4763-b64f-764dacba8a29 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:20:24.731354 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3660ddb4-1a44-4bd3-85bb-e4a1e211053c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5a43c80514b4a13af03c7f8ebcf7a5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:20:24.732253 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3660ddb4-1a44-4bd3-85bb-e4a1e211053c service swift] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:20:24.734822 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3660ddb4-1a44-4bd3-85bb-e4a1e211053c service swift] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:20:24.735325 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3660ddb4-1a44-4bd3-85bb-e4a1e211053c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:20:24.746890 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3660ddb4-1a44-4bd3-85bb-e4a1e211053c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:20:24.748736 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3660ddb4-1a44-4bd3-85bb-e4a1e211053c service swift] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:20:24.760370 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1113/2246] 10.222.0.26 () {60 vars in 1384 bytes} [Thu Jul 14 09:20:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 14 09:20:34.632362 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=3 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:20:34.755576 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=9 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:20:39.214706 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ba7dbc-d041-449c-86ad-2501a6cc783b None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:20:39.268913 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ba7dbc-d041-449c-86ad-2501a6cc783b None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:20:39.273358 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6840d5f7-c05e-4d17-a3ef-b92b708400b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '952911c492444ffea4b211a24b564117', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:20:39.274236 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6840d5f7-c05e-4d17-a3ef-b92b708400b0 service neutron] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:20:39.274439 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6840d5f7-c05e-4d17-a3ef-b92b708400b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:20:39.274735 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6840d5f7-c05e-4d17-a3ef-b92b708400b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:20:39.280410 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-6840d5f7-c05e-4d17-a3ef-b92b708400b0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:20:39.348893 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6840d5f7-c05e-4d17-a3ef-b92b708400b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:20:39.350368 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6840d5f7-c05e-4d17-a3ef-b92b708400b0 service neutron] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:20:39.360189 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1134/2247] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:20:39 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:20:49.355705 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=50 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:20:57.856949 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e924c20d-abbb-4ff0-9274-ce1d77099fc1 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:20:57.857569 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e924c20d-abbb-4ff0-9274-ce1d77099fc1 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:20:57.857569 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e924c20d-abbb-4ff0-9274-ce1d77099fc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:20:57.876082 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-e924c20d-abbb-4ff0-9274-ce1d77099fc1 None None] MFA Rules not processed for user `7d6267ede8d64e2abe1f4058a9797fb7`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:20:57.877286 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-e924c20d-abbb-4ff0-9274-ce1d77099fc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:20:57.884580 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1114/2248] 10.222.0.26 () {58 vars in 1009 bytes} [Thu Jul 14 09:20:57 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:20:57.899926 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6840d5f7-c05e-4d17-a3ef-b92b708400b0 service neutron] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:20:57.911729 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6840d5f7-c05e-4d17-a3ef-b92b708400b0 service neutron] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:20:57.916777 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7057b634-4773-4f85-91a7-54f287138f02 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5a43c80514b4a13af03c7f8ebcf7a5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:20:57.917426 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7057b634-4773-4f85-91a7-54f287138f02 service swift] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:20:57.917709 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7057b634-4773-4f85-91a7-54f287138f02 service swift] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:20:57.917986 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7057b634-4773-4f85-91a7-54f287138f02 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:20:57.927002 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7057b634-4773-4f85-91a7-54f287138f02 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:20:57.928657 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7057b634-4773-4f85-91a7-54f287138f02 service swift] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:20:57.936578 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1135/2249] 10.222.0.26 () {60 vars in 1384 bytes} [Thu Jul 14 09:20:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 14 09:20:58.924279 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e924c20d-abbb-4ff0-9274-ce1d77099fc1 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:20:58.936849 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e924c20d-abbb-4ff0-9274-ce1d77099fc1 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:20:58.941472 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefcaa18-b232-4a0b-9214-66a947098d21 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '952911c492444ffea4b211a24b564117', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:20:58.942386 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefcaa18-b232-4a0b-9214-66a947098d21 service neutron] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:20:58.942862 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefcaa18-b232-4a0b-9214-66a947098d21 service neutron] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:20:58.943434 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefcaa18-b232-4a0b-9214-66a947098d21 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:20:58.954150 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefcaa18-b232-4a0b-9214-66a947098d21 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:20:58.955730 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefcaa18-b232-4a0b-9214-66a947098d21 service neutron] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:20:58.968071 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1115/2250] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:20:58 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:21:07.933794 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=6 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:21:08.963734 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=9 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:21:10.755522 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7057b634-4773-4f85-91a7-54f287138f02 service swift] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:21:10.757211 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-7057b634-4773-4f85-91a7-54f287138f02 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:21:10.841770 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7057b634-4773-4f85-91a7-54f287138f02 service swift] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:21:10.847204 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e598d4e3-daca-41ac-b7db-13406926c4ac service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '355684f52e074ad9bfa7fd90bcae6caf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:21:10.847849 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e598d4e3-daca-41ac-b7db-13406926c4ac service placement] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:21:10.848129 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e598d4e3-daca-41ac-b7db-13406926c4ac service placement] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:21:10.848403 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e598d4e3-daca-41ac-b7db-13406926c4ac service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:21:10.850689 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-e598d4e3-daca-41ac-b7db-13406926c4ac service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:21:10.862572 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e598d4e3-daca-41ac-b7db-13406926c4ac service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:21:10.864193 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e598d4e3-daca-41ac-b7db-13406926c4ac service placement] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:21:10.874437 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1136/2251] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:21:10 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:21:11.874515 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3837d499-725c-44b2-8dd6-06ec4f189227 None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:21:11.875081 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3837d499-725c-44b2-8dd6-06ec4f189227 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:21:11.875081 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3837d499-725c-44b2-8dd6-06ec4f189227 None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:21:11.876643 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1116/2252] 10.222.0.26 () {54 vars in 921 bytes} [Thu Jul 14 09:21:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:21:11.884511 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb241ea0-e8ba-4beb-bfb3-a6f89ce0fcf3 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:21:11.885096 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb241ea0-e8ba-4beb-bfb3-a6f89ce0fcf3 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:21:11.885210 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb241ea0-e8ba-4beb-bfb3-a6f89ce0fcf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:21:11.905424 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-bb241ea0-e8ba-4beb-bfb3-a6f89ce0fcf3 None None] MFA Rules not processed for user `538fff3c5a2b487eb44ae7c1a1198102`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:21:11.907158 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-bb241ea0-e8ba-4beb-bfb3-a6f89ce0fcf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:21:11.993798 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1137/2253] 10.222.0.26 () {58 vars in 1018 bytes} [Thu Jul 14 09:21:11 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:21:12.007921 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3837d499-725c-44b2-8dd6-06ec4f189227 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:21:12.019655 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3837d499-725c-44b2-8dd6-06ec4f189227 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:21:12.023297 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef3c759-7996-40e9-87eb-dd532ee27268 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '872dbb91c9e54f13b1a31fc8242126df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:21:12.023946 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef3c759-7996-40e9-87eb-dd532ee27268 service glance] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:21:12.024230 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef3c759-7996-40e9-87eb-dd532ee27268 service glance] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:21:12.024505 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef3c759-7996-40e9-87eb-dd532ee27268 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:21:12.035501 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef3c759-7996-40e9-87eb-dd532ee27268 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:21:12.036961 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef3c759-7996-40e9-87eb-dd532ee27268 service glance] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:21:12.048430 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1117/2254] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:21:12 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:21:12.119938 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718ebc29-18d0-4b79-b347-db4098b6aa0c None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:21:12.120202 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718ebc29-18d0-4b79-b347-db4098b6aa0c None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:21:12.120491 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718ebc29-18d0-4b79-b347-db4098b6aa0c None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:21:12.121924 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1138/2255] 10.222.0.26 () {54 vars in 921 bytes} [Thu Jul 14 09:21:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:21:12.126907 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c5a518-cd1b-449e-bb06-112ec3a145bf None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:21:12.127215 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c5a518-cd1b-449e-bb06-112ec3a145bf None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:21:12.127535 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c5a518-cd1b-449e-bb06-112ec3a145bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:21:12.145290 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-b8c5a518-cd1b-449e-bb06-112ec3a145bf None None] MFA Rules not processed for user `952911c492444ffea4b211a24b564117`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:21:12.146623 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-b8c5a518-cd1b-449e-bb06-112ec3a145bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:21:12.153540 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1118/2256] 10.222.0.26 () {58 vars in 1018 bytes} [Thu Jul 14 09:21:12 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 09:21:12.165217 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718ebc29-18d0-4b79-b347-db4098b6aa0c None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:21:12.167012 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-718ebc29-18d0-4b79-b347-db4098b6aa0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:21:12.180935 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718ebc29-18d0-4b79-b347-db4098b6aa0c None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:21:12.185015 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196558ca-fa86-4b7b-845c-9da4b8a5e44d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '952911c492444ffea4b211a24b564117', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:21:12.185664 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196558ca-fa86-4b7b-845c-9da4b8a5e44d service neutron] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:21:12.185940 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196558ca-fa86-4b7b-845c-9da4b8a5e44d service neutron] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:21:12.186637 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196558ca-fa86-4b7b-845c-9da4b8a5e44d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:21:12.194251 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196558ca-fa86-4b7b-845c-9da4b8a5e44d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:21:12.195724 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196558ca-fa86-4b7b-845c-9da4b8a5e44d service neutron] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:21:12.204206 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1139/2257] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:21:12 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:21:12.293353 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25738d0-cd9f-4d11-9a9b-586da5715f55 None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:21:12.293645 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25738d0-cd9f-4d11-9a9b-586da5715f55 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:21:12.293926 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25738d0-cd9f-4d11-9a9b-586da5715f55 None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:21:12.295372 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1119/2258] 10.222.0.26 () {54 vars in 921 bytes} [Thu Jul 14 09:21:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:21:12.301338 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041f07a0-dce2-4727-a8d8-b1eb506d0d0c None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:21:12.301625 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041f07a0-dce2-4727-a8d8-b1eb506d0d0c None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:21:12.301967 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041f07a0-dce2-4727-a8d8-b1eb506d0d0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:21:12.321607 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-041f07a0-dce2-4727-a8d8-b1eb506d0d0c None None] MFA Rules not processed for user `538fff3c5a2b487eb44ae7c1a1198102`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:21:12.322787 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-041f07a0-dce2-4727-a8d8-b1eb506d0d0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:21:12.332555 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1140/2259] 10.222.0.26 () {58 vars in 1018 bytes} [Thu Jul 14 09:21:12 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:21:12.343630 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25738d0-cd9f-4d11-9a9b-586da5715f55 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:21:12.355166 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25738d0-cd9f-4d11-9a9b-586da5715f55 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:21:12.359123 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6216d8a8-6465-4df0-8d3e-daf5087827a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '952911c492444ffea4b211a24b564117', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:21:12.359785 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6216d8a8-6465-4df0-8d3e-daf5087827a5 service neutron] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:21:12.360064 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6216d8a8-6465-4df0-8d3e-daf5087827a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:21:12.360334 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6216d8a8-6465-4df0-8d3e-daf5087827a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:21:12.371072 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6216d8a8-6465-4df0-8d3e-daf5087827a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:21:12.372459 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6216d8a8-6465-4df0-8d3e-daf5087827a5 service neutron] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:21:12.383761 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1120/2260] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:21:12 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 09:21:16.686580 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0763f1f-6d66-45f1-b3ed-f9a575ab6d47 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:21:16.687508 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0763f1f-6d66-45f1-b3ed-f9a575ab6d47 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:21:16.687960 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0763f1f-6d66-45f1-b3ed-f9a575ab6d47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:21:16.706306 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-d0763f1f-6d66-45f1-b3ed-f9a575ab6d47 None None] MFA Rules not processed for user `7d6267ede8d64e2abe1f4058a9797fb7`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:21:16.707649 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-d0763f1f-6d66-45f1-b3ed-f9a575ab6d47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:21:16.715646 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1141/2261] 10.222.0.26 () {58 vars in 1009 bytes} [Thu Jul 14 09:21:16 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:21:16.727574 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6216d8a8-6465-4df0-8d3e-daf5087827a5 service neutron] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:21:16.739001 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6216d8a8-6465-4df0-8d3e-daf5087827a5 service neutron] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:21:16.744286 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1de976-0a39-4536-af77-2ddc51b50250 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5a43c80514b4a13af03c7f8ebcf7a5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:21:16.744933 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1de976-0a39-4536-af77-2ddc51b50250 service swift] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:21:16.745216 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1de976-0a39-4536-af77-2ddc51b50250 service swift] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:21:16.745493 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1de976-0a39-4536-af77-2ddc51b50250 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:21:16.754772 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1de976-0a39-4536-af77-2ddc51b50250 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:21:16.756479 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1de976-0a39-4536-af77-2ddc51b50250 service swift] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:21:16.765204 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1121/2262] 10.222.0.26 () {60 vars in 1384 bytes} [Thu Jul 14 09:21:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 14 09:21:26.701225 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=75 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:21:26.761915 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=21 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:21:36.054606 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e487938e-44ad-4efb-b20d-befdc6c967ab None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:21:36.055119 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e487938e-44ad-4efb-b20d-befdc6c967ab None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:21:36.055119 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e487938e-44ad-4efb-b20d-befdc6c967ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:21:36.073481 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-e487938e-44ad-4efb-b20d-befdc6c967ab None None] MFA Rules not processed for user `a2ab766e96024b899a01c540d9bd1322`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:21:36.074676 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-e487938e-44ad-4efb-b20d-befdc6c967ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:21:36.131085 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1142/2263] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:21:36 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:21:39.544386 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172ed1a4-6443-4a26-a828-586c984b3a64 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:21:39.545977 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172ed1a4-6443-4a26-a828-586c984b3a64 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:21:39.546413 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172ed1a4-6443-4a26-a828-586c984b3a64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:21:39.577356 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-172ed1a4-6443-4a26-a828-586c984b3a64 None None] MFA Rules not processed for user `7d6267ede8d64e2abe1f4058a9797fb7`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:21:39.579181 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-172ed1a4-6443-4a26-a828-586c984b3a64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:21:39.592628 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1122/2264] 10.222.0.26 () {58 vars in 967 bytes} [Thu Jul 14 09:21:39 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:21:39.617830 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e487938e-44ad-4efb-b20d-befdc6c967ab None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:21:39.630369 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e487938e-44ad-4efb-b20d-befdc6c967ab None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:21:39.635433 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e03eae4-e51e-4980-ad00-92ad0f13cfbc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5a43c80514b4a13af03c7f8ebcf7a5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:21:39.636079 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e03eae4-e51e-4980-ad00-92ad0f13cfbc service swift] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:21:39.636363 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e03eae4-e51e-4980-ad00-92ad0f13cfbc service swift] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:21:39.636636 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e03eae4-e51e-4980-ad00-92ad0f13cfbc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:21:39.645083 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e03eae4-e51e-4980-ad00-92ad0f13cfbc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:21:39.646662 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e03eae4-e51e-4980-ad00-92ad0f13cfbc service swift] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:21:39.655368 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1143/2265] 10.222.0.26 () {60 vars in 1384 bytes} [Thu Jul 14 09:21:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 14 09:21:49.567257 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=3 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:21:49.651973 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=31 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:21:56.584673 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef5a960-2b22-4a3c-938f-739b8b38d414 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:21:56.585226 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef5a960-2b22-4a3c-938f-739b8b38d414 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:21:56.585226 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef5a960-2b22-4a3c-938f-739b8b38d414 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:21:56.603247 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-1ef5a960-2b22-4a3c-938f-739b8b38d414 None None] MFA Rules not processed for user `18377cf2b2d14cb0b4cbda6ec720ed28`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:21:56.604457 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-1ef5a960-2b22-4a3c-938f-739b8b38d414 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:21:56.612560 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1123/2266] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:21:56 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:22:06.597997 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=3 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:22:31.612108 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9eca05-5c6e-48ae-a6e4-ba9476e97330 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:22:31.613060 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9eca05-5c6e-48ae-a6e4-ba9476e97330 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:22:31.613340 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9eca05-5c6e-48ae-a6e4-ba9476e97330 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:22:31.631349 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-3b9eca05-5c6e-48ae-a6e4-ba9476e97330 None None] MFA Rules not processed for user `4afe1ec7d1ab4e40892d1dde44ce1667`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:22:31.632575 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-3b9eca05-5c6e-48ae-a6e4-ba9476e97330 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:22:31.687898 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1144/2267] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:22:31 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:22:38.819123 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbb53cd-855d-4926-b3c3-a2ad8de5a5e3 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:22:38.820074 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbb53cd-855d-4926-b3c3-a2ad8de5a5e3 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:22:38.820528 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbb53cd-855d-4926-b3c3-a2ad8de5a5e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:22:38.839918 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-1bbb53cd-855d-4926-b3c3-a2ad8de5a5e3 None None] MFA Rules not processed for user `4d619bfc0939493480fa98451db2f30f`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:22:38.841183 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-1bbb53cd-855d-4926-b3c3-a2ad8de5a5e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:22:38.898731 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1124/2268] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:22:38 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:22:40.663543 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9eca05-5c6e-48ae-a6e4-ba9476e97330 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:22:40.673785 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9eca05-5c6e-48ae-a6e4-ba9476e97330 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:22:40.678186 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61390cda-3057-4fb9-af9a-3d1b900d541f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '952911c492444ffea4b211a24b564117', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:22:40.678874 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61390cda-3057-4fb9-af9a-3d1b900d541f service neutron] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:22:40.679126 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61390cda-3057-4fb9-af9a-3d1b900d541f service neutron] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:22:40.679433 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61390cda-3057-4fb9-af9a-3d1b900d541f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:22:40.681571 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-61390cda-3057-4fb9-af9a-3d1b900d541f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:22:40.696025 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61390cda-3057-4fb9-af9a-3d1b900d541f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:22:40.697463 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61390cda-3057-4fb9-af9a-3d1b900d541f service neutron] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:22:40.709774 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1145/2269] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:22:40 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:22:41.631609 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbb53cd-855d-4926-b3c3-a2ad8de5a5e3 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:22:41.633786 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-1bbb53cd-855d-4926-b3c3-a2ad8de5a5e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:22:41.706893 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbb53cd-855d-4926-b3c3-a2ad8de5a5e3 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:22:41.710548 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdab678-44f4-49b3-9869-38233f2876ae service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e76f1952a5b146808844b9f2b7395035', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:22:41.711253 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdab678-44f4-49b3-9869-38233f2876ae service cinder] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:22:41.711540 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdab678-44f4-49b3-9869-38233f2876ae service cinder] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:22:41.711820 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdab678-44f4-49b3-9869-38233f2876ae service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:22:41.713607 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-7fdab678-44f4-49b3-9869-38233f2876ae service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:22:41.727543 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdab678-44f4-49b3-9869-38233f2876ae service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:22:41.729066 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdab678-44f4-49b3-9869-38233f2876ae service cinder] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:22:41.743453 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1125/2270] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:22:41 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 09:22:49.136195 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61390cda-3057-4fb9-af9a-3d1b900d541f service neutron] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:22:49.137798 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-61390cda-3057-4fb9-af9a-3d1b900d541f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:22:49.151887 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61390cda-3057-4fb9-af9a-3d1b900d541f service neutron] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:22:49.158023 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2571bef7-265d-490a-b07d-16f3ae84daea service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '538fff3c5a2b487eb44ae7c1a1198102', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:22:49.158730 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2571bef7-265d-490a-b07d-16f3ae84daea service nova] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:22:49.159085 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2571bef7-265d-490a-b07d-16f3ae84daea service nova] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:22:49.159390 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2571bef7-265d-490a-b07d-16f3ae84daea service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:22:49.161218 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-2571bef7-265d-490a-b07d-16f3ae84daea service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:22:49.171871 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2571bef7-265d-490a-b07d-16f3ae84daea service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:22:49.173456 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2571bef7-265d-490a-b07d-16f3ae84daea service nova] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:22:49.182796 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1146/2271] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:22:49 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:22:51.736454 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=51 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:22:53.125564 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdab678-44f4-49b3-9869-38233f2876ae service cinder] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:22:53.140266 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdab678-44f4-49b3-9869-38233f2876ae service cinder] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:22:53.154386 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01de45e1-2b6d-442c-9de8-30b9c5b7fea2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '538fff3c5a2b487eb44ae7c1a1198102', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:22:53.155783 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01de45e1-2b6d-442c-9de8-30b9c5b7fea2 service nova] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:22:53.156338 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01de45e1-2b6d-442c-9de8-30b9c5b7fea2 service nova] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:22:53.156941 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01de45e1-2b6d-442c-9de8-30b9c5b7fea2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:22:53.159481 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-01de45e1-2b6d-442c-9de8-30b9c5b7fea2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:22:53.182501 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01de45e1-2b6d-442c-9de8-30b9c5b7fea2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:22:53.194657 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01de45e1-2b6d-442c-9de8-30b9c5b7fea2 service nova] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:22:53.215807 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1126/2272] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:22:53 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:22:54.331226 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f32eda-c2be-45a7-aecc-ac5ede4d0c49 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:22:54.331753 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f32eda-c2be-45a7-aecc-ac5ede4d0c49 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:22:54.332158 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f32eda-c2be-45a7-aecc-ac5ede4d0c49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:22:54.362681 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-f6f32eda-c2be-45a7-aecc-ac5ede4d0c49 None None] MFA Rules not processed for user `a2ab766e96024b899a01c540d9bd1322`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:22:54.364054 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-f6f32eda-c2be-45a7-aecc-ac5ede4d0c49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:22:54.371534 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1147/2273] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:22:54 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:22:56.157336 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01de45e1-2b6d-442c-9de8-30b9c5b7fea2 service nova] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:22:56.170358 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01de45e1-2b6d-442c-9de8-30b9c5b7fea2 service nova] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:22:56.174348 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8036cab9-16ef-41a7-a08d-0068f17dd11c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '952911c492444ffea4b211a24b564117', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:22:56.175537 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8036cab9-16ef-41a7-a08d-0068f17dd11c service neutron] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:22:56.176041 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8036cab9-16ef-41a7-a08d-0068f17dd11c service neutron] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:22:56.176546 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8036cab9-16ef-41a7-a08d-0068f17dd11c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:22:56.178878 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-8036cab9-16ef-41a7-a08d-0068f17dd11c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:22:56.192405 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f32eda-c2be-45a7-aecc-ac5ede4d0c49 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:22:56.204086 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8036cab9-16ef-41a7-a08d-0068f17dd11c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:22:56.207538 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8036cab9-16ef-41a7-a08d-0068f17dd11c service neutron] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:22:56.214246 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f32eda-c2be-45a7-aecc-ac5ede4d0c49 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:22:56.220922 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bd38af-a25d-4c78-a3e6-b36a7b431d77 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '538fff3c5a2b487eb44ae7c1a1198102', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:22:56.222722 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bd38af-a25d-4c78-a3e6-b36a7b431d77 service nova] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:22:56.223314 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bd38af-a25d-4c78-a3e6-b36a7b431d77 service nova] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:22:56.223874 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bd38af-a25d-4c78-a3e6-b36a7b431d77 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:22:56.230762 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1127/2274] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:22:56 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:22:56.236645 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bd38af-a25d-4c78-a3e6-b36a7b431d77 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:22:56.238797 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bd38af-a25d-4c78-a3e6-b36a7b431d77 service nova] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:22:56.255007 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1148/2275] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:22:56 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 09:23:05.473999 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b436f7-c201-43f0-adad-aef505f2cd9a None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:23:05.473999 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b436f7-c201-43f0-adad-aef505f2cd9a None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:23:05.473999 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b436f7-c201-43f0-adad-aef505f2cd9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:23:05.492556 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-a1b436f7-c201-43f0-adad-aef505f2cd9a None None] MFA Rules not processed for user `e072ca213ed74cbda810a1fd971ae23d`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:23:05.495730 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-a1b436f7-c201-43f0-adad-aef505f2cd9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:23:05.511998 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1128/2276] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:23:05 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:23:05.856860 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bd38af-a25d-4c78-a3e6-b36a7b431d77 service nova] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:23:05.861556 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=47 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:23:05.867688 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bd38af-a25d-4c78-a3e6-b36a7b431d77 service nova] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:23:05.871539 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c324ecc-af8c-41ed-9ebd-25e1f5483626 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '952911c492444ffea4b211a24b564117', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:23:05.872180 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c324ecc-af8c-41ed-9ebd-25e1f5483626 service neutron] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:23:05.872465 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c324ecc-af8c-41ed-9ebd-25e1f5483626 service neutron] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:23:05.872741 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c324ecc-af8c-41ed-9ebd-25e1f5483626 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:23:05.874676 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-3c324ecc-af8c-41ed-9ebd-25e1f5483626 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:23:05.884350 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c324ecc-af8c-41ed-9ebd-25e1f5483626 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:23:05.885788 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c324ecc-af8c-41ed-9ebd-25e1f5483626 service neutron] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:23:05.893904 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1149/2277] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:23:05 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:23:06.378117 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b436f7-c201-43f0-adad-aef505f2cd9a None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:23:06.380233 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-a1b436f7-c201-43f0-adad-aef505f2cd9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:23:06.399299 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b436f7-c201-43f0-adad-aef505f2cd9a None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:23:06.405480 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3772c68b-f2db-4f9a-b155-c778f82a3461 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '355684f52e074ad9bfa7fd90bcae6caf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:23:06.406702 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3772c68b-f2db-4f9a-b155-c778f82a3461 service placement] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:23:06.407217 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3772c68b-f2db-4f9a-b155-c778f82a3461 service placement] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:23:06.407518 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3772c68b-f2db-4f9a-b155-c778f82a3461 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:23:06.409903 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-3772c68b-f2db-4f9a-b155-c778f82a3461 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:23:06.424426 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3772c68b-f2db-4f9a-b155-c778f82a3461 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:23:06.425626 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3772c68b-f2db-4f9a-b155-c778f82a3461 service placement] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:23:06.438078 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1129/2278] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:23:06 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:23:10.871182 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c324ecc-af8c-41ed-9ebd-25e1f5483626 service neutron] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:23:10.879951 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c324ecc-af8c-41ed-9ebd-25e1f5483626 service neutron] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:23:10.884706 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d100b4d-c37e-4399-a45c-a7fceca6ebe4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '952911c492444ffea4b211a24b564117', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:23:10.885399 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d100b4d-c37e-4399-a45c-a7fceca6ebe4 service neutron] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:23:10.885679 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d100b4d-c37e-4399-a45c-a7fceca6ebe4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:23:10.885951 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d100b4d-c37e-4399-a45c-a7fceca6ebe4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:23:10.888040 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-6d100b4d-c37e-4399-a45c-a7fceca6ebe4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:23:10.899289 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d100b4d-c37e-4399-a45c-a7fceca6ebe4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:23:10.901194 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d100b4d-c37e-4399-a45c-a7fceca6ebe4 service neutron] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:23:10.910850 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1150/2279] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:23:10 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:23:11.572493 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3772c68b-f2db-4f9a-b155-c778f82a3461 service placement] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:23:11.585106 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3772c68b-f2db-4f9a-b155-c778f82a3461 service placement] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:23:11.591636 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484bc529-54f9-4080-870c-92439d2cfdc0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '538fff3c5a2b487eb44ae7c1a1198102', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:23:11.592324 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484bc529-54f9-4080-870c-92439d2cfdc0 service nova] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:23:11.592564 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484bc529-54f9-4080-870c-92439d2cfdc0 service nova] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:23:11.592838 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484bc529-54f9-4080-870c-92439d2cfdc0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:23:11.594741 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-484bc529-54f9-4080-870c-92439d2cfdc0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:23:11.608392 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484bc529-54f9-4080-870c-92439d2cfdc0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:23:11.609993 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484bc529-54f9-4080-870c-92439d2cfdc0 service nova] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:23:11.621683 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1130/2280] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:23:11 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:23:13.849385 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d100b4d-c37e-4399-a45c-a7fceca6ebe4 service neutron] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:23:13.858218 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d100b4d-c37e-4399-a45c-a7fceca6ebe4 service neutron] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:23:13.861835 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1add4de5-70f0-465a-8c1a-6c5b2d032792 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '952911c492444ffea4b211a24b564117', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:23:13.862483 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1add4de5-70f0-465a-8c1a-6c5b2d032792 service neutron] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:23:13.862768 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1add4de5-70f0-465a-8c1a-6c5b2d032792 service neutron] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:23:13.863043 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1add4de5-70f0-465a-8c1a-6c5b2d032792 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:23:13.864858 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-1add4de5-70f0-465a-8c1a-6c5b2d032792 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:23:13.930808 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1add4de5-70f0-465a-8c1a-6c5b2d032792 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:23:13.932325 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1add4de5-70f0-465a-8c1a-6c5b2d032792 service neutron] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:23:13.942186 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1151/2281] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:23:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 09:23:14.826177 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba00b7bd-f15e-4292-8ff1-faf79cf55df0 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:23:14.827490 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba00b7bd-f15e-4292-8ff1-faf79cf55df0 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:23:14.827913 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba00b7bd-f15e-4292-8ff1-faf79cf55df0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:23:14.846883 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-ba00b7bd-f15e-4292-8ff1-faf79cf55df0 None None] MFA Rules not processed for user `7d6267ede8d64e2abe1f4058a9797fb7`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:23:14.848390 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-ba00b7bd-f15e-4292-8ff1-faf79cf55df0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:23:14.856686 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1131/2282] 10.222.0.26 () {58 vars in 967 bytes} [Thu Jul 14 09:23:14 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:23:14.920876 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1add4de5-70f0-465a-8c1a-6c5b2d032792 service neutron] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:23:14.934459 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1add4de5-70f0-465a-8c1a-6c5b2d032792 service neutron] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:23:14.942264 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3605601-21e1-45ba-b169-49b7bb454711 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5a43c80514b4a13af03c7f8ebcf7a5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:23:14.943302 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3605601-21e1-45ba-b169-49b7bb454711 service swift] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:23:14.943736 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3605601-21e1-45ba-b169-49b7bb454711 service swift] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:23:14.944174 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3605601-21e1-45ba-b169-49b7bb454711 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:23:14.955218 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3605601-21e1-45ba-b169-49b7bb454711 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:23:14.956856 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3605601-21e1-45ba-b169-49b7bb454711 service swift] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:23:14.973484 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1152/2283] 10.222.0.26 () {60 vars in 1384 bytes} [Thu Jul 14 09:23:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 14 09:23:19.280360 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba00b7bd-f15e-4292-8ff1-faf79cf55df0 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:23:19.295051 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba00b7bd-f15e-4292-8ff1-faf79cf55df0 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:23:19.300584 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60d444c-b799-4ca7-b03e-803fe2b0b7db service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '355684f52e074ad9bfa7fd90bcae6caf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:23:19.302455 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60d444c-b799-4ca7-b03e-803fe2b0b7db service placement] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:23:19.303421 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60d444c-b799-4ca7-b03e-803fe2b0b7db service placement] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:23:19.303873 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60d444c-b799-4ca7-b03e-803fe2b0b7db service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:23:19.306850 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-d60d444c-b799-4ca7-b03e-803fe2b0b7db service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:23:19.321459 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60d444c-b799-4ca7-b03e-803fe2b0b7db service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:23:19.325049 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60d444c-b799-4ca7-b03e-803fe2b0b7db service placement] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:23:19.351767 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1132/2284] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:23:19 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:23:19.410322 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3605601-21e1-45ba-b169-49b7bb454711 service swift] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:23:19.411815 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-f3605601-21e1-45ba-b169-49b7bb454711 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:23:19.425617 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3605601-21e1-45ba-b169-49b7bb454711 service swift] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:23:19.431472 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efdda3b-662b-4385-975b-ed32eabea12e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:23:19.432367 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efdda3b-662b-4385-975b-ed32eabea12e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:23:19.432671 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efdda3b-662b-4385-975b-ed32eabea12e admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:23:19.432926 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efdda3b-662b-4385-975b-ed32eabea12e admin admin] PATH_INFO: `/v3/users/18377cf2b2d14cb0b4cbda6ec720ed28` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:23:19.434160 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efdda3b-662b-4385-975b-ed32eabea12e admin admin] RBAC: Authorizing `identity:delete_user(user_id=18377cf2b2d14cb0b4cbda6ec720ed28)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:23:19.436351 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efdda3b-662b-4385-975b-ed32eabea12e admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:23:19.486123 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-6efdda3b-662b-4385-975b-ed32eabea12e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '18377cf2b2d14cb0b4cbda6ec720ed28'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:23:19.486845 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-6efdda3b-662b-4385-975b-ed32eabea12e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '18377cf2b2d14cb0b4cbda6ec720ed28'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:23:19.499633 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-6efdda3b-662b-4385-975b-ed32eabea12e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '18377cf2b2d14cb0b4cbda6ec720ed28'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:23:19.508040 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-6efdda3b-662b-4385-975b-ed32eabea12e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '18377cf2b2d14cb0b4cbda6ec720ed28'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:23:19.508710 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-6efdda3b-662b-4385-975b-ed32eabea12e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '18377cf2b2d14cb0b4cbda6ec720ed28'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:23:19.525308 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1153/2285] 10.222.0.26 () {58 vars in 1223 bytes} [Thu Jul 14 09:23:19 2022] DELETE /identity/v3/users/18377cf2b2d14cb0b4cbda6ec720ed28 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:23:20.173592 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60d444c-b799-4ca7-b03e-803fe2b0b7db service placement] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:23:20.192228 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60d444c-b799-4ca7-b03e-803fe2b0b7db service placement] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:23:20.261045 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90e4662-f1a3-41ca-8847-e3d63aa6c6f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:23:20.262638 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90e4662-f1a3-41ca-8847-e3d63aa6c6f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:23:20.263786 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90e4662-f1a3-41ca-8847-e3d63aa6c6f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:23:20.264116 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90e4662-f1a3-41ca-8847-e3d63aa6c6f5 admin admin] PATH_INFO: `/v3/projects/fe8ab23ae4884afc863a58c5e0b96e5b` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:23:20.265545 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90e4662-f1a3-41ca-8847-e3d63aa6c6f5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fe8ab23ae4884afc863a58c5e0b96e5b)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:23:20.268175 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90e4662-f1a3-41ca-8847-e3d63aa6c6f5 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:23:20.327755 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-e90e4662-f1a3-41ca-8847-e3d63aa6c6f5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fe8ab23ae4884afc863a58c5e0b96e5b', 'request_id': 'req-e90e4662-f1a3-41ca-8847-e3d63aa6c6f5'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:23:20.336811 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-e90e4662-f1a3-41ca-8847-e3d63aa6c6f5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fe8ab23ae4884afc863a58c5e0b96e5b', 'request_id': 'req-e90e4662-f1a3-41ca-8847-e3d63aa6c6f5'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:23:20.343839 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-e90e4662-f1a3-41ca-8847-e3d63aa6c6f5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fe8ab23ae4884afc863a58c5e0b96e5b', 'request_id': 'req-e90e4662-f1a3-41ca-8847-e3d63aa6c6f5'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:23:20.358482 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-e90e4662-f1a3-41ca-8847-e3d63aa6c6f5 admin admin] The token cache is being invalidate because project fe8ab23ae4884afc863a58c5e0b96e5b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80381) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 09:23:20.358855 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-e90e4662-f1a3-41ca-8847-e3d63aa6c6f5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:23:20.359566 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-e90e4662-f1a3-41ca-8847-e3d63aa6c6f5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:23:20.361683 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1133/2286] 10.222.0.26 () {58 vars in 1232 bytes} [Thu Jul 14 09:23:20 2022] DELETE /identity/v3/projects/fe8ab23ae4884afc863a58c5e0b96e5b => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:23:20.366500 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a3c111-8d9c-486d-bfa8-cc898900c42c None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:23:20.366938 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a3c111-8d9c-486d-bfa8-cc898900c42c None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:23:20.367339 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a3c111-8d9c-486d-bfa8-cc898900c42c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:23:20.386996 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-87a3c111-8d9c-486d-bfa8-cc898900c42c None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:23:20.388510 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-87a3c111-8d9c-486d-bfa8-cc898900c42c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:23:20.418876 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:23:20.458820 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1154/2287] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:23:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:23:20.466901 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0e53ed-2b1e-471b-b676-dd03e1485e5d None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:23:20.467212 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0e53ed-2b1e-471b-b676-dd03e1485e5d None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:23:20.467501 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0e53ed-2b1e-471b-b676-dd03e1485e5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:23:20.485709 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-6d0e53ed-2b1e-471b-b676-dd03e1485e5d None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:23:20.486917 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-6d0e53ed-2b1e-471b-b676-dd03e1485e5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:23:20.494871 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1134/2288] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:23:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 09:23:20.499281 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a3c111-8d9c-486d-bfa8-cc898900c42c None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:23:20.509145 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a3c111-8d9c-486d-bfa8-cc898900c42c None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:23:20.513787 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec8d843-2865-47e9-bb60-3e0efa960322 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:23:20.514451 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec8d843-2865-47e9-bb60-3e0efa960322 admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:23:20.514761 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec8d843-2865-47e9-bb60-3e0efa960322 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:23:20.515036 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec8d843-2865-47e9-bb60-3e0efa960322 admin admin] PATH_INFO: `/v3/domains` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:23:20.515897 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec8d843-2865-47e9-bb60-3e0efa960322 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:23:20.516370 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec8d843-2865-47e9-bb60-3e0efa960322 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:23:20.518290 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec8d843-2865-47e9-bb60-3e0efa960322 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:23:20.530939 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1155/2289] 10.222.0.26 () {58 vars in 1152 bytes} [Thu Jul 14 09:23:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:23:20.535346 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0e53ed-2b1e-471b-b676-dd03e1485e5d None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:23:20.541460 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:23:20.546264 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0e53ed-2b1e-471b-b676-dd03e1485e5d None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:23:20.554269 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c486317a-6edc-43b5-994e-f95b7869f720 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:23:20.555469 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c486317a-6edc-43b5-994e-f95b7869f720 admin admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:23:20.555750 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c486317a-6edc-43b5-994e-f95b7869f720 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:23:20.556033 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c486317a-6edc-43b5-994e-f95b7869f720 admin admin] PATH_INFO: `/v3/projects` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:23:20.556945 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c486317a-6edc-43b5-994e-f95b7869f720 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:23:20.558962 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c486317a-6edc-43b5-994e-f95b7869f720 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:23:20.586090 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1135/2290] 10.222.0.26 () {60 vars in 1152 bytes} [Thu Jul 14 09:23:20 2022] POST /identity/v3/projects => generated 371 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 09:23:20.590975 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec8d843-2865-47e9-bb60-3e0efa960322 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:23:20.601063 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec8d843-2865-47e9-bb60-3e0efa960322 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:23:20.663926 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4796188b-4578-4ff6-b31f-dbd1293951fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:23:20.664744 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4796188b-4578-4ff6-b31f-dbd1293951fb admin admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:23:20.665029 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4796188b-4578-4ff6-b31f-dbd1293951fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:23:20.665292 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4796188b-4578-4ff6-b31f-dbd1293951fb admin admin] PATH_INFO: `/v3/users` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:23:20.666006 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4796188b-4578-4ff6-b31f-dbd1293951fb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:23:20.667998 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4796188b-4578-4ff6-b31f-dbd1293951fb admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:23:20.688923 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1156/2291] 10.222.0.26 () {60 vars in 1143 bytes} [Thu Jul 14 09:23:20 2022] POST /identity/v3/users => generated 286 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 09:23:20.693136 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c486317a-6edc-43b5-994e-f95b7869f720 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:23:20.703865 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c486317a-6edc-43b5-994e-f95b7869f720 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:23:20.709306 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d182c9b6-c59f-47b3-8408-b0de01286aa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:23:20.710222 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d182c9b6-c59f-47b3-8408-b0de01286aa1 admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:23:20.710759 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d182c9b6-c59f-47b3-8408-b0de01286aa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:23:20.711030 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d182c9b6-c59f-47b3-8408-b0de01286aa1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:23:20.711950 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d182c9b6-c59f-47b3-8408-b0de01286aa1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:23:20.713590 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d182c9b6-c59f-47b3-8408-b0de01286aa1 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:23:20.724651 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1136/2292] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jul 14 09:23:20 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 09:23:20.728849 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4796188b-4578-4ff6-b31f-dbd1293951fb admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:23:20.739314 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4796188b-4578-4ff6-b31f-dbd1293951fb admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:23:20.744691 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b572da0-3d22-4cd5-a0f1-bbe6886d0bc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:23:20.745361 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b572da0-3d22-4cd5-a0f1-bbe6886d0bc2 admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:23:20.745670 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b572da0-3d22-4cd5-a0f1-bbe6886d0bc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:23:20.745946 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b572da0-3d22-4cd5-a0f1-bbe6886d0bc2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:23:20.746678 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b572da0-3d22-4cd5-a0f1-bbe6886d0bc2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:23:20.748319 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b572da0-3d22-4cd5-a0f1-bbe6886d0bc2 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:23:20.758806 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1157/2293] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jul 14 09:23:20 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 09:23:20.763026 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d182c9b6-c59f-47b3-8408-b0de01286aa1 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:23:20.773575 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d182c9b6-c59f-47b3-8408-b0de01286aa1 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:23:20.778385 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67559f7f-8307-4ba4-883d-472f2c6906dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:23:20.779347 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67559f7f-8307-4ba4-883d-472f2c6906dc admin admin] REQUEST_METHOD: `PUT` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:23:20.779636 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67559f7f-8307-4ba4-883d-472f2c6906dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:23:20.779909 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67559f7f-8307-4ba4-883d-472f2c6906dc admin admin] PATH_INFO: `/v3/projects/0914a31f2cb54b9098dc8502b24cd4ab/users/d2c0f919e0c046ad87a28912ebcb9eb5/roles/d1691322ce6a4edab7ce722edf0f2988` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:23:20.795985 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67559f7f-8307-4ba4-883d-472f2c6906dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=0914a31f2cb54b9098dc8502b24cd4ab, user_id=d2c0f919e0c046ad87a28912ebcb9eb5, role_id=d1691322ce6a4edab7ce722edf0f2988)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:23:20.798504 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67559f7f-8307-4ba4-883d-472f2c6906dc admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:23:20.816403 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1137/2294] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jul 14 09:23:20 2022] PUT /identity/v3/projects/0914a31f2cb54b9098dc8502b24cd4ab/users/d2c0f919e0c046ad87a28912ebcb9eb5/roles/d1691322ce6a4edab7ce722edf0f2988 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 09:23:20.828130 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94449f6e-80ca-4506-8edf-2deda85fed24 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:23:20.828433 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94449f6e-80ca-4506-8edf-2deda85fed24 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:23:20.828731 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94449f6e-80ca-4506-8edf-2deda85fed24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:23:20.860193 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-94449f6e-80ca-4506-8edf-2deda85fed24 None None] MFA Rules not processed for user `d2c0f919e0c046ad87a28912ebcb9eb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:23:20.861676 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-94449f6e-80ca-4506-8edf-2deda85fed24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:23:20.951499 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1158/2295] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:23:20 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:23:20.956033 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67559f7f-8307-4ba4-883d-472f2c6906dc admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:23:21.023400 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67559f7f-8307-4ba4-883d-472f2c6906dc admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:23:21.035928 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00782ac0-88d0-488a-b5b2-9ac280b24125 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:23:21.036823 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00782ac0-88d0-488a-b5b2-9ac280b24125 admin admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:23:21.037182 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00782ac0-88d0-488a-b5b2-9ac280b24125 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:23:21.037530 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00782ac0-88d0-488a-b5b2-9ac280b24125 admin admin] PATH_INFO: `/v3/projects` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:23:21.038308 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00782ac0-88d0-488a-b5b2-9ac280b24125 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:23:21.040412 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00782ac0-88d0-488a-b5b2-9ac280b24125 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:23:21.068165 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1138/2296] 10.222.0.26 () {60 vars in 1152 bytes} [Thu Jul 14 09:23:20 2022] POST /identity/v3/projects => generated 371 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 09:23:21.072214 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94449f6e-80ca-4506-8edf-2deda85fed24 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:23:21.083290 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94449f6e-80ca-4506-8edf-2deda85fed24 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:23:21.107239 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:23:21.148907 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d85b20c-9997-4528-b624-f86e20bf8df2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:23:21.149620 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d85b20c-9997-4528-b624-f86e20bf8df2 admin admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:23:21.149909 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d85b20c-9997-4528-b624-f86e20bf8df2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:23:21.150184 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d85b20c-9997-4528-b624-f86e20bf8df2 admin admin] PATH_INFO: `/v3/users` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:23:21.151045 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d85b20c-9997-4528-b624-f86e20bf8df2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:23:21.153114 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d85b20c-9997-4528-b624-f86e20bf8df2 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:23:21.174741 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1159/2297] 10.222.0.26 () {60 vars in 1143 bytes} [Thu Jul 14 09:23:21 2022] POST /identity/v3/users => generated 292 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 09:23:21.178822 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00782ac0-88d0-488a-b5b2-9ac280b24125 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:23:21.192707 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00782ac0-88d0-488a-b5b2-9ac280b24125 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:23:21.197741 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bed2f5-904f-4c88-a90c-5b7f34f81e3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:23:21.198430 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bed2f5-904f-4c88-a90c-5b7f34f81e3a admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:23:21.198715 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bed2f5-904f-4c88-a90c-5b7f34f81e3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:23:21.199013 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bed2f5-904f-4c88-a90c-5b7f34f81e3a admin admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:23:21.199816 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bed2f5-904f-4c88-a90c-5b7f34f81e3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:23:21.201451 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bed2f5-904f-4c88-a90c-5b7f34f81e3a admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:23:21.217475 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1139/2298] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jul 14 09:23:21 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 09:23:21.221871 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d85b20c-9997-4528-b624-f86e20bf8df2 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:23:21.235637 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d85b20c-9997-4528-b624-f86e20bf8df2 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:23:21.241607 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32fc121-a7d3-48b3-b012-22deed9904c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:23:21.242525 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32fc121-a7d3-48b3-b012-22deed9904c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:23:21.242804 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32fc121-a7d3-48b3-b012-22deed9904c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:23:21.243066 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32fc121-a7d3-48b3-b012-22deed9904c9 admin admin] PATH_INFO: `/v3/projects/47c75e4977284fffa799af587dd467d3/users/0b144e6437de4dd9adc890d3f2e68a77/roles/7b549d58d1724395b343609d895c80d0` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:23:21.261846 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32fc121-a7d3-48b3-b012-22deed9904c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=47c75e4977284fffa799af587dd467d3, user_id=0b144e6437de4dd9adc890d3f2e68a77, role_id=7b549d58d1724395b343609d895c80d0)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:23:21.264664 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32fc121-a7d3-48b3-b012-22deed9904c9 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:23:21.281750 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1160/2299] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jul 14 09:23:21 2022] PUT /identity/v3/projects/47c75e4977284fffa799af587dd467d3/users/0b144e6437de4dd9adc890d3f2e68a77/roles/7b549d58d1724395b343609d895c80d0 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:23:21.285911 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bed2f5-904f-4c88-a90c-5b7f34f81e3a admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:23:21.297329 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bed2f5-904f-4c88-a90c-5b7f34f81e3a admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:23:21.365954 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa2a78f-6f02-4107-b3a9-09cdb40ea902 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:23:21.366719 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa2a78f-6f02-4107-b3a9-09cdb40ea902 admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:23:21.366964 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa2a78f-6f02-4107-b3a9-09cdb40ea902 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:23:21.367320 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa2a78f-6f02-4107-b3a9-09cdb40ea902 admin admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:23:21.368025 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa2a78f-6f02-4107-b3a9-09cdb40ea902 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:23:21.369681 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa2a78f-6f02-4107-b3a9-09cdb40ea902 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:23:21.382583 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1140/2300] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jul 14 09:23:21 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 09:23:21.386330 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32fc121-a7d3-48b3-b012-22deed9904c9 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:23:21.398030 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32fc121-a7d3-48b3-b012-22deed9904c9 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:23:21.403022 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c104e789-fa1d-4bbb-96b4-b6711f041f71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:23:21.404133 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c104e789-fa1d-4bbb-96b4-b6711f041f71 admin admin] REQUEST_METHOD: `PUT` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:23:21.404475 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c104e789-fa1d-4bbb-96b4-b6711f041f71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:23:21.404810 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c104e789-fa1d-4bbb-96b4-b6711f041f71 admin admin] PATH_INFO: `/v3/projects/47c75e4977284fffa799af587dd467d3/users/0b144e6437de4dd9adc890d3f2e68a77/roles/7b549d58d1724395b343609d895c80d0` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:23:21.406919 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c104e789-fa1d-4bbb-96b4-b6711f041f71 admin admin] RBAC: Authorizing `identity:create_grant(project_id=47c75e4977284fffa799af587dd467d3, user_id=0b144e6437de4dd9adc890d3f2e68a77, role_id=7b549d58d1724395b343609d895c80d0)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:23:21.409768 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c104e789-fa1d-4bbb-96b4-b6711f041f71 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:23:21.423089 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1161/2301] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jul 14 09:23:21 2022] PUT /identity/v3/projects/47c75e4977284fffa799af587dd467d3/users/0b144e6437de4dd9adc890d3f2e68a77/roles/7b549d58d1724395b343609d895c80d0 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:23:21.434524 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023f6e17-9ae1-47f5-9cbc-63293072499b None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:23:21.435034 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023f6e17-9ae1-47f5-9cbc-63293072499b None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:23:21.436228 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023f6e17-9ae1-47f5-9cbc-63293072499b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:23:21.467800 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-023f6e17-9ae1-47f5-9cbc-63293072499b None None] MFA Rules not processed for user `0b144e6437de4dd9adc890d3f2e68a77`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:23:21.469079 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-023f6e17-9ae1-47f5-9cbc-63293072499b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:23:21.474620 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:23:21.551068 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1141/2302] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:23:21 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:23:21.559592 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c104e789-fa1d-4bbb-96b4-b6711f041f71 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:23:21.561677 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-c104e789-fa1d-4bbb-96b4-b6711f041f71 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:23:21.624678 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c104e789-fa1d-4bbb-96b4-b6711f041f71 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:23:21.628531 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d12d184-387f-4c82-b564-1a2ffbb9fd7c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e76f1952a5b146808844b9f2b7395035', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:23:21.629187 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d12d184-387f-4c82-b564-1a2ffbb9fd7c service cinder] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:23:21.629467 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d12d184-387f-4c82-b564-1a2ffbb9fd7c service cinder] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:23:21.629776 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d12d184-387f-4c82-b564-1a2ffbb9fd7c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:23:21.641080 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d12d184-387f-4c82-b564-1a2ffbb9fd7c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:23:21.642534 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d12d184-387f-4c82-b564-1a2ffbb9fd7c service cinder] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:23:21.655306 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1162/2303] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:23:21 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:23:22.057678 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023f6e17-9ae1-47f5-9cbc-63293072499b None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:23:22.153146 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023f6e17-9ae1-47f5-9cbc-63293072499b None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:23:22.163094 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a2bf25-48e0-46c0-9325-647d05e85790 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:23:22.164113 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a2bf25-48e0-46c0-9325-647d05e85790 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:23:22.164404 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a2bf25-48e0-46c0-9325-647d05e85790 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:23:22.165176 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a2bf25-48e0-46c0-9325-647d05e85790 admin admin] PATH_INFO: `/v3/users/d2c0f919e0c046ad87a28912ebcb9eb5` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:23:22.166032 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a2bf25-48e0-46c0-9325-647d05e85790 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d2c0f919e0c046ad87a28912ebcb9eb5)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:23:22.168166 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a2bf25-48e0-46c0-9325-647d05e85790 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:23:22.225268 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-29a2bf25-48e0-46c0-9325-647d05e85790 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd2c0f919e0c046ad87a28912ebcb9eb5'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:23:22.235919 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-29a2bf25-48e0-46c0-9325-647d05e85790 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd2c0f919e0c046ad87a28912ebcb9eb5'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:23:22.237026 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-29a2bf25-48e0-46c0-9325-647d05e85790 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd2c0f919e0c046ad87a28912ebcb9eb5'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:23:22.237568 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-29a2bf25-48e0-46c0-9325-647d05e85790 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd2c0f919e0c046ad87a28912ebcb9eb5'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:23:22.247506 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-29a2bf25-48e0-46c0-9325-647d05e85790 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd2c0f919e0c046ad87a28912ebcb9eb5'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:23:22.268137 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1142/2304] 10.222.0.26 () {58 vars in 1223 bytes} [Thu Jul 14 09:23:22 2022] DELETE /identity/v3/users/d2c0f919e0c046ad87a28912ebcb9eb5 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:23:22.273333 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e4c4a2-954b-42c3-8633-e58c790d67db None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:23:22.273726 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e4c4a2-954b-42c3-8633-e58c790d67db None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:23:22.274091 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e4c4a2-954b-42c3-8633-e58c790d67db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:23:22.295059 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-b4e4c4a2-954b-42c3-8633-e58c790d67db None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:23:22.303675 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-b4e4c4a2-954b-42c3-8633-e58c790d67db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:23:22.380415 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1163/2305] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:23:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:23:22.390343 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a2bf25-48e0-46c0-9325-647d05e85790 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:23:22.392229 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-29a2bf25-48e0-46c0-9325-647d05e85790 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:23:22.422506 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:23:22.460025 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a2bf25-48e0-46c0-9325-647d05e85790 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:23:22.463993 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739eeb72-c952-40a5-90ed-01149da387fc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '952911c492444ffea4b211a24b564117', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:23:22.464730 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739eeb72-c952-40a5-90ed-01149da387fc service neutron] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:23:22.465199 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739eeb72-c952-40a5-90ed-01149da387fc service neutron] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:23:22.465565 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739eeb72-c952-40a5-90ed-01149da387fc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:23:22.475935 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739eeb72-c952-40a5-90ed-01149da387fc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:23:22.477474 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739eeb72-c952-40a5-90ed-01149da387fc service neutron] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:23:22.489382 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1143/2306] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:23:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 09:23:22.791920 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e4c4a2-954b-42c3-8633-e58c790d67db None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:23:22.794141 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-b4e4c4a2-954b-42c3-8633-e58c790d67db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:23:22.812570 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e4c4a2-954b-42c3-8633-e58c790d67db None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:23:22.817363 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf41084-1052-4916-9fb6-68c84806bf49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:23:22.818239 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf41084-1052-4916-9fb6-68c84806bf49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:23:22.818583 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf41084-1052-4916-9fb6-68c84806bf49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:23:22.819367 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf41084-1052-4916-9fb6-68c84806bf49 admin admin] PATH_INFO: `/v3/projects/0914a31f2cb54b9098dc8502b24cd4ab` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:23:22.820866 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf41084-1052-4916-9fb6-68c84806bf49 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0914a31f2cb54b9098dc8502b24cd4ab)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:23:22.822731 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf41084-1052-4916-9fb6-68c84806bf49 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:23:22.840693 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:23:22.878193 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-aaf41084-1052-4916-9fb6-68c84806bf49 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0914a31f2cb54b9098dc8502b24cd4ab', 'request_id': 'req-aaf41084-1052-4916-9fb6-68c84806bf49'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:23:22.884766 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-aaf41084-1052-4916-9fb6-68c84806bf49 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0914a31f2cb54b9098dc8502b24cd4ab', 'request_id': 'req-aaf41084-1052-4916-9fb6-68c84806bf49'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:23:22.890565 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-aaf41084-1052-4916-9fb6-68c84806bf49 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0914a31f2cb54b9098dc8502b24cd4ab', 'request_id': 'req-aaf41084-1052-4916-9fb6-68c84806bf49'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:23:22.904248 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-aaf41084-1052-4916-9fb6-68c84806bf49 admin admin] The token cache is being invalidate because project 0914a31f2cb54b9098dc8502b24cd4ab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80382) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 09:23:22.904542 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-aaf41084-1052-4916-9fb6-68c84806bf49 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:23:22.905560 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-aaf41084-1052-4916-9fb6-68c84806bf49 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:23:22.907417 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1164/2307] 10.222.0.26 () {58 vars in 1232 bytes} [Thu Jul 14 09:23:22 2022] DELETE /identity/v3/projects/0914a31f2cb54b9098dc8502b24cd4ab => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:23:22.911238 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739eeb72-c952-40a5-90ed-01149da387fc service neutron] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:23:22.923180 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739eeb72-c952-40a5-90ed-01149da387fc service neutron] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:23:22.988107 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b514ae6d-8edf-4e7f-ba62-610e21c48626 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:23:22.989128 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b514ae6d-8edf-4e7f-ba62-610e21c48626 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:23:22.989467 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b514ae6d-8edf-4e7f-ba62-610e21c48626 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:23:22.989802 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b514ae6d-8edf-4e7f-ba62-610e21c48626 admin admin] PATH_INFO: `/v3/users/0b144e6437de4dd9adc890d3f2e68a77` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:23:22.991693 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b514ae6d-8edf-4e7f-ba62-610e21c48626 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0b144e6437de4dd9adc890d3f2e68a77)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:23:22.993776 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b514ae6d-8edf-4e7f-ba62-610e21c48626 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:23:23.058017 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-b514ae6d-8edf-4e7f-ba62-610e21c48626 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0b144e6437de4dd9adc890d3f2e68a77'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:23:23.066192 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-b514ae6d-8edf-4e7f-ba62-610e21c48626 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0b144e6437de4dd9adc890d3f2e68a77'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:23:23.066997 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-b514ae6d-8edf-4e7f-ba62-610e21c48626 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0b144e6437de4dd9adc890d3f2e68a77'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:23:23.067707 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-b514ae6d-8edf-4e7f-ba62-610e21c48626 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0b144e6437de4dd9adc890d3f2e68a77'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:23:23.075317 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-b514ae6d-8edf-4e7f-ba62-610e21c48626 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0b144e6437de4dd9adc890d3f2e68a77'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:23:23.093711 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1144/2308] 10.222.0.26 () {58 vars in 1223 bytes} [Thu Jul 14 09:23:22 2022] DELETE /identity/v3/users/0b144e6437de4dd9adc890d3f2e68a77 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:23:23.417635 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf41084-1052-4916-9fb6-68c84806bf49 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:23:23.419177 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-aaf41084-1052-4916-9fb6-68c84806bf49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:23:23.447285 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf41084-1052-4916-9fb6-68c84806bf49 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:23:23.564072 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882b0b79-bcb1-4edd-87b9-1e368fc31ad5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:23:23.565200 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882b0b79-bcb1-4edd-87b9-1e368fc31ad5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:23:23.565636 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882b0b79-bcb1-4edd-87b9-1e368fc31ad5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:23:23.566102 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882b0b79-bcb1-4edd-87b9-1e368fc31ad5 admin admin] PATH_INFO: `/v3/projects/47c75e4977284fffa799af587dd467d3` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:23:23.567386 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882b0b79-bcb1-4edd-87b9-1e368fc31ad5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=47c75e4977284fffa799af587dd467d3)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:23:23.570129 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882b0b79-bcb1-4edd-87b9-1e368fc31ad5 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:23:23.643364 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-882b0b79-bcb1-4edd-87b9-1e368fc31ad5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '47c75e4977284fffa799af587dd467d3', 'request_id': 'req-882b0b79-bcb1-4edd-87b9-1e368fc31ad5'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:23:23.653424 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-882b0b79-bcb1-4edd-87b9-1e368fc31ad5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '47c75e4977284fffa799af587dd467d3', 'request_id': 'req-882b0b79-bcb1-4edd-87b9-1e368fc31ad5'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:23:23.666860 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-882b0b79-bcb1-4edd-87b9-1e368fc31ad5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '47c75e4977284fffa799af587dd467d3', 'request_id': 'req-882b0b79-bcb1-4edd-87b9-1e368fc31ad5'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:23:23.710051 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-882b0b79-bcb1-4edd-87b9-1e368fc31ad5 admin admin] The token cache is being invalidate because project 47c75e4977284fffa799af587dd467d3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80382) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 09:23:23.710051 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-882b0b79-bcb1-4edd-87b9-1e368fc31ad5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:23:23.710772 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-882b0b79-bcb1-4edd-87b9-1e368fc31ad5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:23:23.712673 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1165/2309] 10.222.0.26 () {58 vars in 1232 bytes} [Thu Jul 14 09:23:23 2022] DELETE /identity/v3/projects/47c75e4977284fffa799af587dd467d3 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 09:23:23.719227 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8320de4a-ff8f-4fd3-99fd-82d5e3dc03be None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:23:23.720180 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8320de4a-ff8f-4fd3-99fd-82d5e3dc03be None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:23:23.720610 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8320de4a-ff8f-4fd3-99fd-82d5e3dc03be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:23:23.750854 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-8320de4a-ff8f-4fd3-99fd-82d5e3dc03be None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:23:23.752799 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-8320de4a-ff8f-4fd3-99fd-82d5e3dc03be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:23:23.764593 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882b0b79-bcb1-4edd-87b9-1e368fc31ad5 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:23:23.766265 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-882b0b79-bcb1-4edd-87b9-1e368fc31ad5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:23:23.815696 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:23:23.841390 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:23:23.873884 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882b0b79-bcb1-4edd-87b9-1e368fc31ad5 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:23:23.875376 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1145/2310] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:23:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:23:23.884071 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1318e7-2080-415f-8575-8ea1cba514b7 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:23:23.884237 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69725a8d-be06-49d4-b8b2-c59a22dab6bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:23:23.884369 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1318e7-2080-415f-8575-8ea1cba514b7 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:23:23.884677 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1318e7-2080-415f-8575-8ea1cba514b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:23:23.885232 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69725a8d-be06-49d4-b8b2-c59a22dab6bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:23:23.885518 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69725a8d-be06-49d4-b8b2-c59a22dab6bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:23:23.885813 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69725a8d-be06-49d4-b8b2-c59a22dab6bf admin admin] PATH_INFO: `/v3/users/b4dcd1f4a61643f69b81e7edae238fd2` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:23:23.887048 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69725a8d-be06-49d4-b8b2-c59a22dab6bf admin admin] RBAC: Authorizing `identity:delete_user(user_id=b4dcd1f4a61643f69b81e7edae238fd2)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:23:23.889115 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69725a8d-be06-49d4-b8b2-c59a22dab6bf admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:23:23.909554 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-8b1318e7-2080-415f-8575-8ea1cba514b7 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:23:23.910818 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-8b1318e7-2080-415f-8575-8ea1cba514b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:23:23.919202 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1146/2311] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:23:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 09:23:23.923753 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1318e7-2080-415f-8575-8ea1cba514b7 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:23:23.933697 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1318e7-2080-415f-8575-8ea1cba514b7 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:23:23.938358 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820256d8-2f99-4bf6-bd13-11990863ff6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:23:23.939015 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820256d8-2f99-4bf6-bd13-11990863ff6c admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:23:23.939344 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820256d8-2f99-4bf6-bd13-11990863ff6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:23:23.939627 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820256d8-2f99-4bf6-bd13-11990863ff6c admin admin] PATH_INFO: `/v3/domains` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:23:23.940353 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820256d8-2f99-4bf6-bd13-11990863ff6c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:23:23.940863 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820256d8-2f99-4bf6-bd13-11990863ff6c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:23:23.942265 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820256d8-2f99-4bf6-bd13-11990863ff6c admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:23:23.946050 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-69725a8d-be06-49d4-b8b2-c59a22dab6bf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b4dcd1f4a61643f69b81e7edae238fd2'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:23:23.946709 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-69725a8d-be06-49d4-b8b2-c59a22dab6bf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b4dcd1f4a61643f69b81e7edae238fd2'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:23:23.954024 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1147/2312] 10.222.0.26 () {58 vars in 1152 bytes} [Thu Jul 14 09:23:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 09:23:23.958364 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820256d8-2f99-4bf6-bd13-11990863ff6c admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:23:23.959057 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-69725a8d-be06-49d4-b8b2-c59a22dab6bf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b4dcd1f4a61643f69b81e7edae238fd2'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:23:23.960062 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-820256d8-2f99-4bf6-bd13-11990863ff6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:23:23.968430 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-69725a8d-be06-49d4-b8b2-c59a22dab6bf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b4dcd1f4a61643f69b81e7edae238fd2'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:23:23.969159 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-69725a8d-be06-49d4-b8b2-c59a22dab6bf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b4dcd1f4a61643f69b81e7edae238fd2'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:23:23.973253 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820256d8-2f99-4bf6-bd13-11990863ff6c admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:23:23.977977 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bccbe4-e5bf-4951-be75-897e69b230ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:23:23.978575 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bccbe4-e5bf-4951-be75-897e69b230ce admin admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:23:23.978854 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bccbe4-e5bf-4951-be75-897e69b230ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:23:23.979129 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bccbe4-e5bf-4951-be75-897e69b230ce admin admin] PATH_INFO: `/v3/projects` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:23:23.979862 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bccbe4-e5bf-4951-be75-897e69b230ce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:23:23.981766 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bccbe4-e5bf-4951-be75-897e69b230ce admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:23:23.982494 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1166/2313] 10.222.0.26 () {58 vars in 1223 bytes} [Thu Jul 14 09:23:23 2022] DELETE /identity/v3/users/b4dcd1f4a61643f69b81e7edae238fd2 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:23:24.009286 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1148/2314] 10.222.0.26 () {60 vars in 1152 bytes} [Thu Jul 14 09:23:23 2022] POST /identity/v3/projects => generated 363 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 09:23:24.017648 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69725a8d-be06-49d4-b8b2-c59a22dab6bf admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:23:24.086581 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69725a8d-be06-49d4-b8b2-c59a22dab6bf admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:23:24.164095 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0e5470-9261-4720-b59c-6870c761ea43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:23:24.164852 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0e5470-9261-4720-b59c-6870c761ea43 admin admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:23:24.165099 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0e5470-9261-4720-b59c-6870c761ea43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:23:24.165780 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0e5470-9261-4720-b59c-6870c761ea43 admin admin] PATH_INFO: `/v3/users` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:23:24.166052 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0e5470-9261-4720-b59c-6870c761ea43 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:23:24.168230 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0e5470-9261-4720-b59c-6870c761ea43 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:23:24.170258 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bccbe4-e5bf-4951-be75-897e69b230ce admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:23:24.172336 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-26bccbe4-e5bf-4951-be75-897e69b230ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:23:24.176642 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:23:24.188076 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bccbe4-e5bf-4951-be75-897e69b230ce admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:23:24.194630 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b792d3-9c4c-46ff-9245-27fa58bdc0b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:23:24.195586 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b792d3-9c4c-46ff-9245-27fa58bdc0b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:23:24.195897 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b792d3-9c4c-46ff-9245-27fa58bdc0b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:23:24.196241 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b792d3-9c4c-46ff-9245-27fa58bdc0b9 admin admin] PATH_INFO: `/v3/projects/e6ebe73241894d12a03f7fde53207274` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:23:24.197612 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b792d3-9c4c-46ff-9245-27fa58bdc0b9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e6ebe73241894d12a03f7fde53207274)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:23:24.199612 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b792d3-9c4c-46ff-9245-27fa58bdc0b9 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:23:24.207881 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1167/2315] 10.222.0.26 () {60 vars in 1143 bytes} [Thu Jul 14 09:23:24 2022] POST /identity/v3/users => generated 282 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 09:23:24.210312 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0e5470-9261-4720-b59c-6870c761ea43 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:23:24.222091 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0e5470-9261-4720-b59c-6870c761ea43 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:23:24.227548 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24a30c9-2589-4c12-9c06-710846fff374 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:23:24.228401 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24a30c9-2589-4c12-9c06-710846fff374 admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:23:24.228825 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24a30c9-2589-4c12-9c06-710846fff374 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:23:24.229197 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24a30c9-2589-4c12-9c06-710846fff374 admin admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:23:24.230165 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24a30c9-2589-4c12-9c06-710846fff374 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:23:24.231816 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24a30c9-2589-4c12-9c06-710846fff374 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:23:24.243782 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1168/2316] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jul 14 09:23:24 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 09:23:24.247413 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24a30c9-2589-4c12-9c06-710846fff374 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:23:24.261065 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24a30c9-2589-4c12-9c06-710846fff374 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:23:24.272339 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-d1b792d3-9c4c-46ff-9245-27fa58bdc0b9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e6ebe73241894d12a03f7fde53207274', 'request_id': 'req-d1b792d3-9c4c-46ff-9245-27fa58bdc0b9'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:23:24.281951 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-d1b792d3-9c4c-46ff-9245-27fa58bdc0b9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e6ebe73241894d12a03f7fde53207274', 'request_id': 'req-d1b792d3-9c4c-46ff-9245-27fa58bdc0b9'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:23:24.289295 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-d1b792d3-9c4c-46ff-9245-27fa58bdc0b9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e6ebe73241894d12a03f7fde53207274', 'request_id': 'req-d1b792d3-9c4c-46ff-9245-27fa58bdc0b9'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:23:24.306727 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-d1b792d3-9c4c-46ff-9245-27fa58bdc0b9 admin admin] The token cache is being invalidate because project e6ebe73241894d12a03f7fde53207274 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80381) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 09:23:24.307142 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-d1b792d3-9c4c-46ff-9245-27fa58bdc0b9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:23:24.308276 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-d1b792d3-9c4c-46ff-9245-27fa58bdc0b9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:23:24.310077 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1149/2317] 10.222.0.26 () {58 vars in 1232 bytes} [Thu Jul 14 09:23:24 2022] DELETE /identity/v3/projects/e6ebe73241894d12a03f7fde53207274 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 09:23:24.314019 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b792d3-9c4c-46ff-9245-27fa58bdc0b9 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:23:24.315893 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-d1b792d3-9c4c-46ff-9245-27fa58bdc0b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:23:24.323422 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79fbc72-fc34-4a45-a650-564d9e29e29a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:23:24.324190 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79fbc72-fc34-4a45-a650-564d9e29e29a admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:23:24.324540 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79fbc72-fc34-4a45-a650-564d9e29e29a admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:23:24.324904 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79fbc72-fc34-4a45-a650-564d9e29e29a admin admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:23:24.325706 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79fbc72-fc34-4a45-a650-564d9e29e29a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:23:24.327467 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79fbc72-fc34-4a45-a650-564d9e29e29a admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:23:24.339009 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1169/2318] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jul 14 09:23:24 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 09:23:24.343242 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79fbc72-fc34-4a45-a650-564d9e29e29a admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:23:24.344744 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-e79fbc72-fc34-4a45-a650-564d9e29e29a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:23:24.358872 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79fbc72-fc34-4a45-a650-564d9e29e29a admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:23:24.364668 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff555b3-95af-45b5-a99b-f62233a0808d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:23:24.365693 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff555b3-95af-45b5-a99b-f62233a0808d admin admin] REQUEST_METHOD: `PUT` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:23:24.366192 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff555b3-95af-45b5-a99b-f62233a0808d admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:23:24.366707 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff555b3-95af-45b5-a99b-f62233a0808d admin admin] PATH_INFO: `/v3/projects/1a9ac9d02c004ab0a8cfeb08c2d516bf/users/ec87f15820fd4e5386fd42c481a11052/roles/d1691322ce6a4edab7ce722edf0f2988` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:23:24.386062 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff555b3-95af-45b5-a99b-f62233a0808d admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a9ac9d02c004ab0a8cfeb08c2d516bf, user_id=ec87f15820fd4e5386fd42c481a11052, role_id=d1691322ce6a4edab7ce722edf0f2988)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:23:24.389113 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff555b3-95af-45b5-a99b-f62233a0808d admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:23:24.391749 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b792d3-9c4c-46ff-9245-27fa58bdc0b9 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:23:24.398345 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdf77be-d5e8-403f-ac40-2904470521d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:23:24.399413 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdf77be-d5e8-403f-ac40-2904470521d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:23:24.399694 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdf77be-d5e8-403f-ac40-2904470521d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:23:24.399970 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdf77be-d5e8-403f-ac40-2904470521d8 admin admin] PATH_INFO: `/v3/users/43c335fb11c04f839050207590c31e92` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:23:24.401210 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdf77be-d5e8-403f-ac40-2904470521d8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=43c335fb11c04f839050207590c31e92)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:23:24.403162 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdf77be-d5e8-403f-ac40-2904470521d8 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:23:24.406302 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1170/2319] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jul 14 09:23:24 2022] PUT /identity/v3/projects/1a9ac9d02c004ab0a8cfeb08c2d516bf/users/ec87f15820fd4e5386fd42c481a11052/roles/d1691322ce6a4edab7ce722edf0f2988 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 09:23:24.417895 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91273766-807d-4ea2-b3c4-4a69a3480434 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:23:24.418202 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91273766-807d-4ea2-b3c4-4a69a3480434 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:23:24.418482 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91273766-807d-4ea2-b3c4-4a69a3480434 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:23:24.440835 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=86,keystone:INSERT=2,keystone:DELETE=12 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:23:24.448033 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-91273766-807d-4ea2-b3c4-4a69a3480434 None None] MFA Rules not processed for user `ec87f15820fd4e5386fd42c481a11052`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:23:24.449255 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-91273766-807d-4ea2-b3c4-4a69a3480434 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:23:24.466797 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-3cdf77be-d5e8-403f-ac40-2904470521d8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '43c335fb11c04f839050207590c31e92'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:23:24.474881 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-3cdf77be-d5e8-403f-ac40-2904470521d8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '43c335fb11c04f839050207590c31e92'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:23:24.475659 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-3cdf77be-d5e8-403f-ac40-2904470521d8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '43c335fb11c04f839050207590c31e92'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:23:24.476312 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-3cdf77be-d5e8-403f-ac40-2904470521d8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '43c335fb11c04f839050207590c31e92'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:23:24.483415 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-3cdf77be-d5e8-403f-ac40-2904470521d8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '43c335fb11c04f839050207590c31e92'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:23:24.502048 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1150/2320] 10.222.0.26 () {58 vars in 1223 bytes} [Thu Jul 14 09:23:24 2022] DELETE /identity/v3/users/43c335fb11c04f839050207590c31e92 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 09:23:24.517553 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1171/2321] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:23:24 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:23:24.522366 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdf77be-d5e8-403f-ac40-2904470521d8 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:23:24.523890 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-3cdf77be-d5e8-403f-ac40-2904470521d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:23:24.600571 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdf77be-d5e8-403f-ac40-2904470521d8 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:23:24.605840 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b766b40-845b-4d35-aef6-e9d98e6fc61c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:23:24.606527 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b766b40-845b-4d35-aef6-e9d98e6fc61c admin admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:23:24.606812 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b766b40-845b-4d35-aef6-e9d98e6fc61c admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:23:24.607657 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b766b40-845b-4d35-aef6-e9d98e6fc61c admin admin] PATH_INFO: `/v3/projects` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:23:24.607871 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b766b40-845b-4d35-aef6-e9d98e6fc61c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:23:24.609771 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b766b40-845b-4d35-aef6-e9d98e6fc61c admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:23:24.646996 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1151/2322] 10.222.0.26 () {60 vars in 1152 bytes} [Thu Jul 14 09:23:24 2022] POST /identity/v3/projects => generated 365 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 09:23:24.648401 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91273766-807d-4ea2-b3c4-4a69a3480434 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:23:24.689669 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b766b40-845b-4d35-aef6-e9d98e6fc61c admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:23:24.691788 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-4b766b40-845b-4d35-aef6-e9d98e6fc61c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:23:24.718784 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:23:24.755034 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91273766-807d-4ea2-b3c4-4a69a3480434 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:23:24.759999 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882a37d8-a5fe-44d7-9e77-a3d5bb8a5c21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:23:24.760715 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882a37d8-a5fe-44d7-9e77-a3d5bb8a5c21 admin admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:23:24.760996 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882a37d8-a5fe-44d7-9e77-a3d5bb8a5c21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:23:24.761419 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882a37d8-a5fe-44d7-9e77-a3d5bb8a5c21 admin admin] PATH_INFO: `/v3/users` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:23:24.763344 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882a37d8-a5fe-44d7-9e77-a3d5bb8a5c21 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:23:24.765317 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882a37d8-a5fe-44d7-9e77-a3d5bb8a5c21 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:23:24.766172 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b766b40-845b-4d35-aef6-e9d98e6fc61c admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:23:24.770811 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3f67da-2f5a-4af1-8cfd-bbe9f14655bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:23:24.772196 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3f67da-2f5a-4af1-8cfd-bbe9f14655bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:23:24.772783 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3f67da-2f5a-4af1-8cfd-bbe9f14655bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:23:24.773269 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3f67da-2f5a-4af1-8cfd-bbe9f14655bc admin admin] PATH_INFO: `/v3/projects/9b56c67e6cac415eb089c52f9cc0bf9c` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:23:24.775133 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3f67da-2f5a-4af1-8cfd-bbe9f14655bc admin admin] RBAC: Authorizing `identity:delete_project(project_id=9b56c67e6cac415eb089c52f9cc0bf9c)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:23:24.777391 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3f67da-2f5a-4af1-8cfd-bbe9f14655bc admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:23:24.791273 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1172/2323] 10.222.0.26 () {60 vars in 1143 bytes} [Thu Jul 14 09:23:24 2022] POST /identity/v3/users => generated 289 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 09:23:24.792961 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882a37d8-a5fe-44d7-9e77-a3d5bb8a5c21 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:23:24.802930 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882a37d8-a5fe-44d7-9e77-a3d5bb8a5c21 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:23:24.808365 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c4d1fc-81cc-4533-bb38-4ddb27081b1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:23:24.809083 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c4d1fc-81cc-4533-bb38-4ddb27081b1c admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:23:24.809348 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c4d1fc-81cc-4533-bb38-4ddb27081b1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:23:24.809626 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c4d1fc-81cc-4533-bb38-4ddb27081b1c admin admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:23:24.810381 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c4d1fc-81cc-4533-bb38-4ddb27081b1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:23:24.812035 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c4d1fc-81cc-4533-bb38-4ddb27081b1c admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:23:24.814965 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:23:24.822891 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1173/2324] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jul 14 09:23:24 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 09:23:24.827149 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c4d1fc-81cc-4533-bb38-4ddb27081b1c admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:23:24.828627 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-ca3f67da-2f5a-4af1-8cfd-bbe9f14655bc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9b56c67e6cac415eb089c52f9cc0bf9c', 'request_id': 'req-ca3f67da-2f5a-4af1-8cfd-bbe9f14655bc'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:23:24.835927 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-ca3f67da-2f5a-4af1-8cfd-bbe9f14655bc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9b56c67e6cac415eb089c52f9cc0bf9c', 'request_id': 'req-ca3f67da-2f5a-4af1-8cfd-bbe9f14655bc'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:23:24.836915 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c4d1fc-81cc-4533-bb38-4ddb27081b1c admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:23:24.841504 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-ca3f67da-2f5a-4af1-8cfd-bbe9f14655bc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9b56c67e6cac415eb089c52f9cc0bf9c', 'request_id': 'req-ca3f67da-2f5a-4af1-8cfd-bbe9f14655bc'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:23:24.856119 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-ca3f67da-2f5a-4af1-8cfd-bbe9f14655bc admin admin] The token cache is being invalidate because project 9b56c67e6cac415eb089c52f9cc0bf9c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80381) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 09:23:24.856406 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-ca3f67da-2f5a-4af1-8cfd-bbe9f14655bc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:23:24.857037 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-ca3f67da-2f5a-4af1-8cfd-bbe9f14655bc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:23:24.858605 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1152/2325] 10.222.0.26 () {58 vars in 1232 bytes} [Thu Jul 14 09:23:24 2022] DELETE /identity/v3/projects/9b56c67e6cac415eb089c52f9cc0bf9c => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:23:24.895595 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5e9f7b-89d4-4162-ba60-1f4f838c35f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:23:24.896976 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5e9f7b-89d4-4162-ba60-1f4f838c35f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:23:24.897247 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5e9f7b-89d4-4162-ba60-1f4f838c35f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:23:24.897522 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5e9f7b-89d4-4162-ba60-1f4f838c35f0 admin admin] PATH_INFO: `/v3/projects/9c6cf061e0f848248ea7400cd49ab9de/users/659f00079b1948f7bf154d8f92a6ff12/roles/7b549d58d1724395b343609d895c80d0` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:23:24.914472 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5e9f7b-89d4-4162-ba60-1f4f838c35f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c6cf061e0f848248ea7400cd49ab9de, user_id=659f00079b1948f7bf154d8f92a6ff12, role_id=7b549d58d1724395b343609d895c80d0)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:23:24.917203 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5e9f7b-89d4-4162-ba60-1f4f838c35f0 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:23:24.932749 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1174/2326] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jul 14 09:23:24 2022] PUT /identity/v3/projects/9c6cf061e0f848248ea7400cd49ab9de/users/659f00079b1948f7bf154d8f92a6ff12/roles/7b549d58d1724395b343609d895c80d0 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:23:24.937730 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3f67da-2f5a-4af1-8cfd-bbe9f14655bc admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:23:24.939621 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-ca3f67da-2f5a-4af1-8cfd-bbe9f14655bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:23:24.954469 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3f67da-2f5a-4af1-8cfd-bbe9f14655bc admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:23:25.025445 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3528007-4f28-4ce4-aa27-8cd77627ce24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:23:25.026188 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3528007-4f28-4ce4-aa27-8cd77627ce24 admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:23:25.026750 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3528007-4f28-4ce4-aa27-8cd77627ce24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:23:25.027032 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3528007-4f28-4ce4-aa27-8cd77627ce24 admin admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:23:25.027867 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3528007-4f28-4ce4-aa27-8cd77627ce24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:23:25.029541 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3528007-4f28-4ce4-aa27-8cd77627ce24 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:23:25.041077 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1153/2327] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jul 14 09:23:24 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 09:23:25.045425 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5e9f7b-89d4-4162-ba60-1f4f838c35f0 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:23:25.057036 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5e9f7b-89d4-4162-ba60-1f4f838c35f0 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:23:25.063453 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634ab591-0dd8-4d17-8fbe-21ed3824fcf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:23:25.064556 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634ab591-0dd8-4d17-8fbe-21ed3824fcf3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:23:25.064809 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634ab591-0dd8-4d17-8fbe-21ed3824fcf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:23:25.065078 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634ab591-0dd8-4d17-8fbe-21ed3824fcf3 admin admin] PATH_INFO: `/v3/projects/9c6cf061e0f848248ea7400cd49ab9de/users/659f00079b1948f7bf154d8f92a6ff12/roles/7b549d58d1724395b343609d895c80d0` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:23:25.068093 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634ab591-0dd8-4d17-8fbe-21ed3824fcf3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c6cf061e0f848248ea7400cd49ab9de, user_id=659f00079b1948f7bf154d8f92a6ff12, role_id=7b549d58d1724395b343609d895c80d0)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:23:25.070968 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634ab591-0dd8-4d17-8fbe-21ed3824fcf3 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:23:25.083327 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1175/2328] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jul 14 09:23:25 2022] PUT /identity/v3/projects/9c6cf061e0f848248ea7400cd49ab9de/users/659f00079b1948f7bf154d8f92a6ff12/roles/7b549d58d1724395b343609d895c80d0 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:23:25.094283 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8374de39-c428-4e43-a9b8-8a53829b504b None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:23:25.094573 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8374de39-c428-4e43-a9b8-8a53829b504b None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:23:25.094856 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8374de39-c428-4e43-a9b8-8a53829b504b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:23:25.126717 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-8374de39-c428-4e43-a9b8-8a53829b504b None None] MFA Rules not processed for user `659f00079b1948f7bf154d8f92a6ff12`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:23:25.128058 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-8374de39-c428-4e43-a9b8-8a53829b504b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:23:25.195937 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1154/2329] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:23:25 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:23:25.207506 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634ab591-0dd8-4d17-8fbe-21ed3824fcf3 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:23:25.209626 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-634ab591-0dd8-4d17-8fbe-21ed3824fcf3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:23:25.273967 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634ab591-0dd8-4d17-8fbe-21ed3824fcf3 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:23:25.279948 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9688acda-907f-4a5f-bfeb-23c2cd62d7d2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e76f1952a5b146808844b9f2b7395035', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:23:25.280535 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9688acda-907f-4a5f-bfeb-23c2cd62d7d2 service cinder] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:23:25.280821 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9688acda-907f-4a5f-bfeb-23c2cd62d7d2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:23:25.281107 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9688acda-907f-4a5f-bfeb-23c2cd62d7d2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:23:25.292520 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9688acda-907f-4a5f-bfeb-23c2cd62d7d2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:23:25.293975 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9688acda-907f-4a5f-bfeb-23c2cd62d7d2 service cinder] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:23:25.304619 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1176/2330] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:23:25 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:23:25.383257 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8374de39-c428-4e43-a9b8-8a53829b504b None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:23:25.393370 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8374de39-c428-4e43-a9b8-8a53829b504b None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:23:25.397618 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b269495-a51d-41f6-9fbf-65c066ce6690 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e76f1952a5b146808844b9f2b7395035', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:23:25.398257 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b269495-a51d-41f6-9fbf-65c066ce6690 service cinder] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:23:25.398546 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b269495-a51d-41f6-9fbf-65c066ce6690 service cinder] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:23:25.398850 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b269495-a51d-41f6-9fbf-65c066ce6690 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:23:25.400754 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-7b269495-a51d-41f6-9fbf-65c066ce6690 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:23:25.444857 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:23:25.466006 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b269495-a51d-41f6-9fbf-65c066ce6690 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:23:25.467483 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b269495-a51d-41f6-9fbf-65c066ce6690 service cinder] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:23:25.476816 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1155/2331] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:23:25 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:23:28.221799 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9688acda-907f-4a5f-bfeb-23c2cd62d7d2 service cinder] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:23:28.258536 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:23:28.291306 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9688acda-907f-4a5f-bfeb-23c2cd62d7d2 service cinder] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:23:28.296064 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71423412-5daa-4eec-bf74-73174657f694 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:23:28.297048 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71423412-5daa-4eec-bf74-73174657f694 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:23:28.297315 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71423412-5daa-4eec-bf74-73174657f694 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:23:28.297591 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71423412-5daa-4eec-bf74-73174657f694 admin admin] PATH_INFO: `/v3/users/ec87f15820fd4e5386fd42c481a11052` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:23:28.298801 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71423412-5daa-4eec-bf74-73174657f694 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ec87f15820fd4e5386fd42c481a11052)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:23:28.300819 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71423412-5daa-4eec-bf74-73174657f694 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:23:28.355451 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-71423412-5daa-4eec-bf74-73174657f694 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ec87f15820fd4e5386fd42c481a11052'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:23:28.356069 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-71423412-5daa-4eec-bf74-73174657f694 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ec87f15820fd4e5386fd42c481a11052'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:23:28.368449 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-71423412-5daa-4eec-bf74-73174657f694 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ec87f15820fd4e5386fd42c481a11052'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:23:28.376494 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-71423412-5daa-4eec-bf74-73174657f694 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ec87f15820fd4e5386fd42c481a11052'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:23:28.377062 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-71423412-5daa-4eec-bf74-73174657f694 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ec87f15820fd4e5386fd42c481a11052'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:23:28.390088 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1177/2332] 10.222.0.26 () {58 vars in 1223 bytes} [Thu Jul 14 09:23:28 2022] DELETE /identity/v3/users/ec87f15820fd4e5386fd42c481a11052 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:23:28.394828 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ffb32f-b805-4334-bb96-ae85553c3636 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:23:28.395373 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ffb32f-b805-4334-bb96-ae85553c3636 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:23:28.395479 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ffb32f-b805-4334-bb96-ae85553c3636 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:23:28.413911 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-47ffb32f-b805-4334-bb96-ae85553c3636 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:23:28.415107 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-47ffb32f-b805-4334-bb96-ae85553c3636 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:23:28.477388 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1156/2333] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:23:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:23:28.486901 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71423412-5daa-4eec-bf74-73174657f694 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:23:28.488351 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-71423412-5daa-4eec-bf74-73174657f694 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:23:28.551413 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71423412-5daa-4eec-bf74-73174657f694 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:23:28.557146 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8da4d7-99cd-43e8-8545-6c5e768a4e71 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '952911c492444ffea4b211a24b564117', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:23:28.557817 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8da4d7-99cd-43e8-8545-6c5e768a4e71 service neutron] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:23:28.558101 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8da4d7-99cd-43e8-8545-6c5e768a4e71 service neutron] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:23:28.558375 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8da4d7-99cd-43e8-8545-6c5e768a4e71 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:23:28.568004 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8da4d7-99cd-43e8-8545-6c5e768a4e71 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:23:28.569470 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8da4d7-99cd-43e8-8545-6c5e768a4e71 service neutron] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:23:28.579569 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1178/2334] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:23:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:23:29.007520 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ffb32f-b805-4334-bb96-ae85553c3636 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:23:29.009308 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-47ffb32f-b805-4334-bb96-ae85553c3636 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:23:29.027546 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ffb32f-b805-4334-bb96-ae85553c3636 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:23:29.032720 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd50212-6640-4fbe-8f95-390e7911bdac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:23:29.033642 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd50212-6640-4fbe-8f95-390e7911bdac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:23:29.033926 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd50212-6640-4fbe-8f95-390e7911bdac admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:23:29.034200 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd50212-6640-4fbe-8f95-390e7911bdac admin admin] PATH_INFO: `/v3/projects/1a9ac9d02c004ab0a8cfeb08c2d516bf` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:23:29.035561 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd50212-6640-4fbe-8f95-390e7911bdac admin admin] RBAC: Authorizing `identity:delete_project(project_id=1a9ac9d02c004ab0a8cfeb08c2d516bf)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:23:29.037537 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd50212-6640-4fbe-8f95-390e7911bdac admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:23:29.089328 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-4bd50212-6640-4fbe-8f95-390e7911bdac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1a9ac9d02c004ab0a8cfeb08c2d516bf', 'request_id': 'req-4bd50212-6640-4fbe-8f95-390e7911bdac'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:23:29.095608 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-4bd50212-6640-4fbe-8f95-390e7911bdac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1a9ac9d02c004ab0a8cfeb08c2d516bf', 'request_id': 'req-4bd50212-6640-4fbe-8f95-390e7911bdac'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:23:29.103590 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-4bd50212-6640-4fbe-8f95-390e7911bdac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1a9ac9d02c004ab0a8cfeb08c2d516bf', 'request_id': 'req-4bd50212-6640-4fbe-8f95-390e7911bdac'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:23:29.119121 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-4bd50212-6640-4fbe-8f95-390e7911bdac admin admin] The token cache is being invalidate because project 1a9ac9d02c004ab0a8cfeb08c2d516bf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80381) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 09:23:29.119438 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-4bd50212-6640-4fbe-8f95-390e7911bdac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:23:29.120095 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-4bd50212-6640-4fbe-8f95-390e7911bdac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:23:29.122024 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1157/2335] 10.222.0.26 () {58 vars in 1232 bytes} [Thu Jul 14 09:23:29 2022] DELETE /identity/v3/projects/1a9ac9d02c004ab0a8cfeb08c2d516bf => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 09:23:29.126088 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8da4d7-99cd-43e8-8545-6c5e768a4e71 service neutron] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:23:29.135841 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8da4d7-99cd-43e8-8545-6c5e768a4e71 service neutron] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:23:29.185430 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:23:29.197661 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a18d20a-99d6-410a-8c2e-fe657ec8b8a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:23:29.198607 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a18d20a-99d6-410a-8c2e-fe657ec8b8a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:23:29.198907 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a18d20a-99d6-410a-8c2e-fe657ec8b8a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:23:29.199397 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a18d20a-99d6-410a-8c2e-fe657ec8b8a5 admin admin] PATH_INFO: `/v3/users/659f00079b1948f7bf154d8f92a6ff12` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:23:29.200604 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a18d20a-99d6-410a-8c2e-fe657ec8b8a5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=659f00079b1948f7bf154d8f92a6ff12)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:23:29.203163 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a18d20a-99d6-410a-8c2e-fe657ec8b8a5 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:23:29.249849 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-2a18d20a-99d6-410a-8c2e-fe657ec8b8a5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '659f00079b1948f7bf154d8f92a6ff12'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:23:29.250538 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-2a18d20a-99d6-410a-8c2e-fe657ec8b8a5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '659f00079b1948f7bf154d8f92a6ff12'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:23:29.263153 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-2a18d20a-99d6-410a-8c2e-fe657ec8b8a5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '659f00079b1948f7bf154d8f92a6ff12'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:23:29.271028 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-2a18d20a-99d6-410a-8c2e-fe657ec8b8a5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '659f00079b1948f7bf154d8f92a6ff12'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:23:29.272011 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-2a18d20a-99d6-410a-8c2e-fe657ec8b8a5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '659f00079b1948f7bf154d8f92a6ff12'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:23:29.284858 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1179/2336] 10.222.0.26 () {58 vars in 1223 bytes} [Thu Jul 14 09:23:29 2022] DELETE /identity/v3/users/659f00079b1948f7bf154d8f92a6ff12 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 09:23:29.584476 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd50212-6640-4fbe-8f95-390e7911bdac admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:23:29.587013 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-4bd50212-6640-4fbe-8f95-390e7911bdac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:23:29.619597 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd50212-6640-4fbe-8f95-390e7911bdac admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:23:29.695877 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577165ac-795c-4a6d-b681-052e34fe645e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:23:29.697133 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577165ac-795c-4a6d-b681-052e34fe645e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:23:29.697576 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577165ac-795c-4a6d-b681-052e34fe645e admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:23:29.697780 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577165ac-795c-4a6d-b681-052e34fe645e admin admin] PATH_INFO: `/v3/projects/9c6cf061e0f848248ea7400cd49ab9de` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:23:29.699518 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577165ac-795c-4a6d-b681-052e34fe645e admin admin] RBAC: Authorizing `identity:delete_project(project_id=9c6cf061e0f848248ea7400cd49ab9de)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:23:29.702141 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577165ac-795c-4a6d-b681-052e34fe645e admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:23:29.710639 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:23:29.767460 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-577165ac-795c-4a6d-b681-052e34fe645e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9c6cf061e0f848248ea7400cd49ab9de', 'request_id': 'req-577165ac-795c-4a6d-b681-052e34fe645e'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:23:29.775448 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-577165ac-795c-4a6d-b681-052e34fe645e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9c6cf061e0f848248ea7400cd49ab9de', 'request_id': 'req-577165ac-795c-4a6d-b681-052e34fe645e'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:23:29.783120 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-577165ac-795c-4a6d-b681-052e34fe645e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9c6cf061e0f848248ea7400cd49ab9de', 'request_id': 'req-577165ac-795c-4a6d-b681-052e34fe645e'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:23:29.799604 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-577165ac-795c-4a6d-b681-052e34fe645e admin admin] The token cache is being invalidate because project 9c6cf061e0f848248ea7400cd49ab9de was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80381) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 09:23:29.800539 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-577165ac-795c-4a6d-b681-052e34fe645e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:23:29.801859 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-577165ac-795c-4a6d-b681-052e34fe645e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:23:29.804415 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1158/2337] 10.222.0.26 () {58 vars in 1232 bytes} [Thu Jul 14 09:23:29 2022] DELETE /identity/v3/projects/9c6cf061e0f848248ea7400cd49ab9de => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:23:29.809676 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846dadbf-05c5-42e9-aa54-db25eb0dd76b None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:23:29.810013 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846dadbf-05c5-42e9-aa54-db25eb0dd76b None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:23:29.810293 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846dadbf-05c5-42e9-aa54-db25eb0dd76b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:23:29.829394 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-846dadbf-05c5-42e9-aa54-db25eb0dd76b None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:23:29.830593 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-846dadbf-05c5-42e9-aa54-db25eb0dd76b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:23:29.892008 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1180/2338] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:23:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:23:29.901506 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822600e3-960a-4fc7-b6f0-d107f26dc820 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:23:29.901991 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822600e3-960a-4fc7-b6f0-d107f26dc820 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:23:29.902346 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822600e3-960a-4fc7-b6f0-d107f26dc820 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:23:29.921407 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-822600e3-960a-4fc7-b6f0-d107f26dc820 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:23:29.922604 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-822600e3-960a-4fc7-b6f0-d107f26dc820 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:23:29.930557 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1159/2339] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:23:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:23:29.934916 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846dadbf-05c5-42e9-aa54-db25eb0dd76b None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:23:29.944761 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846dadbf-05c5-42e9-aa54-db25eb0dd76b None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:23:29.949449 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2825cac-e610-41f5-a363-dbbe6cd2dc03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:23:29.950113 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2825cac-e610-41f5-a363-dbbe6cd2dc03 admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:23:29.950398 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2825cac-e610-41f5-a363-dbbe6cd2dc03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:23:29.950671 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2825cac-e610-41f5-a363-dbbe6cd2dc03 admin admin] PATH_INFO: `/v3/domains` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:23:29.956069 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2825cac-e610-41f5-a363-dbbe6cd2dc03 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:23:29.956389 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2825cac-e610-41f5-a363-dbbe6cd2dc03 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:23:29.958426 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2825cac-e610-41f5-a363-dbbe6cd2dc03 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:23:29.973913 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1181/2340] 10.222.0.26 () {58 vars in 1152 bytes} [Thu Jul 14 09:23:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:23:29.978164 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822600e3-960a-4fc7-b6f0-d107f26dc820 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:23:29.987522 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822600e3-960a-4fc7-b6f0-d107f26dc820 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:23:29.992454 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78252f4-ec5d-4aa5-b0c1-8df2c532a817 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:23:29.993068 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78252f4-ec5d-4aa5-b0c1-8df2c532a817 admin admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:23:29.993356 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78252f4-ec5d-4aa5-b0c1-8df2c532a817 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:23:29.993633 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78252f4-ec5d-4aa5-b0c1-8df2c532a817 admin admin] PATH_INFO: `/v3/projects` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:23:29.994349 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78252f4-ec5d-4aa5-b0c1-8df2c532a817 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:23:29.996318 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78252f4-ec5d-4aa5-b0c1-8df2c532a817 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:23:30.027316 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1160/2341] 10.222.0.26 () {60 vars in 1152 bytes} [Thu Jul 14 09:23:29 2022] POST /identity/v3/projects => generated 379 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 09:23:30.031839 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2825cac-e610-41f5-a363-dbbe6cd2dc03 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:23:30.042090 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2825cac-e610-41f5-a363-dbbe6cd2dc03 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:23:30.104307 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616ab507-9125-48d7-ab28-ad8aa62541b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:23:30.105035 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616ab507-9125-48d7-ab28-ad8aa62541b9 admin admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:23:30.105347 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616ab507-9125-48d7-ab28-ad8aa62541b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:23:30.105658 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616ab507-9125-48d7-ab28-ad8aa62541b9 admin admin] PATH_INFO: `/v3/users` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:23:30.106416 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616ab507-9125-48d7-ab28-ad8aa62541b9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:23:30.108478 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616ab507-9125-48d7-ab28-ad8aa62541b9 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:23:30.115559 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:23:30.132847 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1182/2342] 10.222.0.26 () {60 vars in 1143 bytes} [Thu Jul 14 09:23:30 2022] POST /identity/v3/users => generated 290 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 09:23:30.136903 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78252f4-ec5d-4aa5-b0c1-8df2c532a817 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:23:30.148408 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78252f4-ec5d-4aa5-b0c1-8df2c532a817 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:23:30.153338 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5eb2ad6-6110-4d80-aa51-814790159323 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:23:30.154027 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5eb2ad6-6110-4d80-aa51-814790159323 admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:23:30.154345 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5eb2ad6-6110-4d80-aa51-814790159323 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:23:30.154665 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5eb2ad6-6110-4d80-aa51-814790159323 admin admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:23:30.155482 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5eb2ad6-6110-4d80-aa51-814790159323 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:23:30.157125 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5eb2ad6-6110-4d80-aa51-814790159323 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:23:30.168258 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1161/2343] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jul 14 09:23:30 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 09:23:30.172563 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616ab507-9125-48d7-ab28-ad8aa62541b9 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:23:30.183535 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616ab507-9125-48d7-ab28-ad8aa62541b9 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:23:30.188854 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42fbfc3-8f95-41d1-b74e-b35be1944c39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:23:30.189531 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42fbfc3-8f95-41d1-b74e-b35be1944c39 admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:23:30.189819 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42fbfc3-8f95-41d1-b74e-b35be1944c39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:23:30.190092 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42fbfc3-8f95-41d1-b74e-b35be1944c39 admin admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:23:30.190829 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42fbfc3-8f95-41d1-b74e-b35be1944c39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:23:30.192537 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42fbfc3-8f95-41d1-b74e-b35be1944c39 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:23:30.203582 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1183/2344] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jul 14 09:23:30 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 09:23:30.207951 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5eb2ad6-6110-4d80-aa51-814790159323 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:23:30.218760 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5eb2ad6-6110-4d80-aa51-814790159323 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:23:30.224148 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6502b2a9-e36a-4c4a-8d9f-61778f358cb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:23:30.225050 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6502b2a9-e36a-4c4a-8d9f-61778f358cb0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:23:30.225339 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6502b2a9-e36a-4c4a-8d9f-61778f358cb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:23:30.225690 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6502b2a9-e36a-4c4a-8d9f-61778f358cb0 admin admin] PATH_INFO: `/v3/projects/e33c6a5d82dd4ba89a58a86e219b66c5/users/f0d6e1639747400899eaa8e4e03ce39c/roles/d1691322ce6a4edab7ce722edf0f2988` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:23:30.245816 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6502b2a9-e36a-4c4a-8d9f-61778f358cb0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e33c6a5d82dd4ba89a58a86e219b66c5, user_id=f0d6e1639747400899eaa8e4e03ce39c, role_id=d1691322ce6a4edab7ce722edf0f2988)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:23:30.249346 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6502b2a9-e36a-4c4a-8d9f-61778f358cb0 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:23:30.267574 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1162/2345] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jul 14 09:23:30 2022] PUT /identity/v3/projects/e33c6a5d82dd4ba89a58a86e219b66c5/users/f0d6e1639747400899eaa8e4e03ce39c/roles/d1691322ce6a4edab7ce722edf0f2988 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:23:30.278548 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01260d7d-2ec1-41ab-9bbe-c0d8f9f882e9 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:23:30.279136 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01260d7d-2ec1-41ab-9bbe-c0d8f9f882e9 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:23:30.279320 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01260d7d-2ec1-41ab-9bbe-c0d8f9f882e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:23:30.307359 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6502b2a9-e36a-4c4a-8d9f-61778f358cb0 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:23:30.309254 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-01260d7d-2ec1-41ab-9bbe-c0d8f9f882e9 None None] MFA Rules not processed for user `f0d6e1639747400899eaa8e4e03ce39c`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:23:30.309389 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-6502b2a9-e36a-4c4a-8d9f-61778f358cb0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:23:30.310716 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-01260d7d-2ec1-41ab-9bbe-c0d8f9f882e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:23:30.395092 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1184/2346] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:23:30 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:23:30.397119 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01260d7d-2ec1-41ab-9bbe-c0d8f9f882e9 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:23:30.397530 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6502b2a9-e36a-4c4a-8d9f-61778f358cb0 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:23:30.403538 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2635902b-b07c-4ea2-826a-c6d3bcacc967 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '538fff3c5a2b487eb44ae7c1a1198102', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:23:30.404290 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2635902b-b07c-4ea2-826a-c6d3bcacc967 service nova] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:23:30.404662 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2635902b-b07c-4ea2-826a-c6d3bcacc967 service nova] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:23:30.405025 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2635902b-b07c-4ea2-826a-c6d3bcacc967 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:23:30.406895 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-2635902b-b07c-4ea2-826a-c6d3bcacc967 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:23:30.420358 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2635902b-b07c-4ea2-826a-c6d3bcacc967 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:23:30.422857 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2635902b-b07c-4ea2-826a-c6d3bcacc967 service nova] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:23:30.437413 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1163/2347] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:23:30 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:23:30.479830 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01260d7d-2ec1-41ab-9bbe-c0d8f9f882e9 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:23:30.484382 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12da0508-d810-46f4-ba45-f6fc9fd0f978 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:23:30.485048 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12da0508-d810-46f4-ba45-f6fc9fd0f978 admin admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:23:30.485359 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12da0508-d810-46f4-ba45-f6fc9fd0f978 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:23:30.485750 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12da0508-d810-46f4-ba45-f6fc9fd0f978 admin admin] PATH_INFO: `/v3/projects` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:23:30.486455 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12da0508-d810-46f4-ba45-f6fc9fd0f978 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:23:30.488402 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12da0508-d810-46f4-ba45-f6fc9fd0f978 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:23:30.520411 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2635902b-b07c-4ea2-826a-c6d3bcacc967 service nova] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:23:30.522777 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1185/2348] 10.222.0.26 () {60 vars in 1152 bytes} [Thu Jul 14 09:23:30 2022] POST /identity/v3/projects => generated 379 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 09:23:30.533300 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2635902b-b07c-4ea2-826a-c6d3bcacc967 service nova] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:23:30.544867 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:23:30.597017 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0101b74-1544-4f1b-abcc-0a987da6fe43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:23:30.597808 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0101b74-1544-4f1b-abcc-0a987da6fe43 admin admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:23:30.598156 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0101b74-1544-4f1b-abcc-0a987da6fe43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:23:30.598495 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0101b74-1544-4f1b-abcc-0a987da6fe43 admin admin] PATH_INFO: `/v3/users` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:23:30.599295 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0101b74-1544-4f1b-abcc-0a987da6fe43 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:23:30.601544 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0101b74-1544-4f1b-abcc-0a987da6fe43 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:23:30.625815 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1164/2349] 10.222.0.26 () {60 vars in 1143 bytes} [Thu Jul 14 09:23:30 2022] POST /identity/v3/users => generated 290 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 09:23:30.629661 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12da0508-d810-46f4-ba45-f6fc9fd0f978 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:23:30.641824 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12da0508-d810-46f4-ba45-f6fc9fd0f978 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:23:30.647680 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6122fdb8-1595-4835-ae2d-d1cfbca03a12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:23:30.648388 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6122fdb8-1595-4835-ae2d-d1cfbca03a12 admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:23:30.648772 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6122fdb8-1595-4835-ae2d-d1cfbca03a12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:23:30.649225 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6122fdb8-1595-4835-ae2d-d1cfbca03a12 admin admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:23:30.650084 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6122fdb8-1595-4835-ae2d-d1cfbca03a12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:23:30.651929 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6122fdb8-1595-4835-ae2d-d1cfbca03a12 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:23:30.664716 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1186/2350] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jul 14 09:23:30 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 09:23:30.669901 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0101b74-1544-4f1b-abcc-0a987da6fe43 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:23:30.684381 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0101b74-1544-4f1b-abcc-0a987da6fe43 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:23:30.689125 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c92b250-8c82-4e15-aa4d-d9cf19c4321c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:23:30.689803 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c92b250-8c82-4e15-aa4d-d9cf19c4321c admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:23:30.690091 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c92b250-8c82-4e15-aa4d-d9cf19c4321c admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:23:30.690517 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c92b250-8c82-4e15-aa4d-d9cf19c4321c admin admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:23:30.691300 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c92b250-8c82-4e15-aa4d-d9cf19c4321c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:23:30.692923 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c92b250-8c82-4e15-aa4d-d9cf19c4321c admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:23:30.703894 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1165/2351] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jul 14 09:23:30 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 09:23:30.708698 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6122fdb8-1595-4835-ae2d-d1cfbca03a12 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:23:30.723687 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6122fdb8-1595-4835-ae2d-d1cfbca03a12 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:23:30.729061 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a64596e-4cea-476b-9f06-1376d0828144 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:23:30.730504 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a64596e-4cea-476b-9f06-1376d0828144 admin admin] REQUEST_METHOD: `PUT` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:23:30.730857 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a64596e-4cea-476b-9f06-1376d0828144 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:23:30.731255 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a64596e-4cea-476b-9f06-1376d0828144 admin admin] PATH_INFO: `/v3/projects/b02fa012cca342278d7a5330a4dc012f/users/06400fbd4bd94ee58be6c457ec6e789a/roles/d1691322ce6a4edab7ce722edf0f2988` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:23:30.750449 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a64596e-4cea-476b-9f06-1376d0828144 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b02fa012cca342278d7a5330a4dc012f, user_id=06400fbd4bd94ee58be6c457ec6e789a, role_id=d1691322ce6a4edab7ce722edf0f2988)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:23:30.753198 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a64596e-4cea-476b-9f06-1376d0828144 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:23:30.774292 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1187/2352] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jul 14 09:23:30 2022] PUT /identity/v3/projects/b02fa012cca342278d7a5330a4dc012f/users/06400fbd4bd94ee58be6c457ec6e789a/roles/d1691322ce6a4edab7ce722edf0f2988 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 09:23:30.787783 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d4c4c8-231d-44bd-b2d9-9a2374164e72 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:23:30.788249 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d4c4c8-231d-44bd-b2d9-9a2374164e72 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:23:30.788620 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d4c4c8-231d-44bd-b2d9-9a2374164e72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:23:30.825034 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-59d4c4c8-231d-44bd-b2d9-9a2374164e72 None None] MFA Rules not processed for user `06400fbd4bd94ee58be6c457ec6e789a`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:23:30.826349 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-59d4c4c8-231d-44bd-b2d9-9a2374164e72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:23:30.906641 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1166/2353] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:23:30 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:23:30.911080 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a64596e-4cea-476b-9f06-1376d0828144 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:23:30.933268 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:23:30.980264 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a64596e-4cea-476b-9f06-1376d0828144 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:23:30.985321 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956a9f96-f71e-434f-ad70-55fa6c642f6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:23:30.986055 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956a9f96-f71e-434f-ad70-55fa6c642f6a admin admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:23:30.986507 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956a9f96-f71e-434f-ad70-55fa6c642f6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:23:30.986792 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956a9f96-f71e-434f-ad70-55fa6c642f6a admin admin] PATH_INFO: `/v3/projects` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:23:30.987693 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956a9f96-f71e-434f-ad70-55fa6c642f6a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:23:30.989803 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956a9f96-f71e-434f-ad70-55fa6c642f6a admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:23:31.019919 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1188/2354] 10.222.0.26 () {60 vars in 1152 bytes} [Thu Jul 14 09:23:30 2022] POST /identity/v3/projects => generated 377 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 09:23:31.024878 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d4c4c8-231d-44bd-b2d9-9a2374164e72 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:23:31.045580 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d4c4c8-231d-44bd-b2d9-9a2374164e72 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:23:31.107875 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ef6e5f-24b8-4609-9dd1-5e6aa8541731 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:23:31.108564 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ef6e5f-24b8-4609-9dd1-5e6aa8541731 admin admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:23:31.108852 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ef6e5f-24b8-4609-9dd1-5e6aa8541731 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:23:31.109133 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ef6e5f-24b8-4609-9dd1-5e6aa8541731 admin admin] PATH_INFO: `/v3/users` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:23:31.109872 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ef6e5f-24b8-4609-9dd1-5e6aa8541731 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:23:31.112160 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ef6e5f-24b8-4609-9dd1-5e6aa8541731 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:23:31.135303 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1167/2355] 10.222.0.26 () {60 vars in 1143 bytes} [Thu Jul 14 09:23:31 2022] POST /identity/v3/users => generated 295 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 09:23:31.139130 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956a9f96-f71e-434f-ad70-55fa6c642f6a admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:23:31.150777 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956a9f96-f71e-434f-ad70-55fa6c642f6a admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:23:31.155879 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0549aac-9860-4ec1-9900-c7c2de0101a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:23:31.156589 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0549aac-9860-4ec1-9900-c7c2de0101a3 admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:23:31.156931 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0549aac-9860-4ec1-9900-c7c2de0101a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:23:31.157146 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0549aac-9860-4ec1-9900-c7c2de0101a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:23:31.157908 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0549aac-9860-4ec1-9900-c7c2de0101a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:23:31.159674 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0549aac-9860-4ec1-9900-c7c2de0101a3 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:23:31.171256 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1189/2356] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jul 14 09:23:31 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 09:23:31.175629 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ef6e5f-24b8-4609-9dd1-5e6aa8541731 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:23:31.186776 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ef6e5f-24b8-4609-9dd1-5e6aa8541731 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:23:31.192013 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f604d89-9e7d-40ef-8c07-80e6abd0eda5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:23:31.192931 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f604d89-9e7d-40ef-8c07-80e6abd0eda5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:23:31.193354 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f604d89-9e7d-40ef-8c07-80e6abd0eda5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:23:31.193641 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f604d89-9e7d-40ef-8c07-80e6abd0eda5 admin admin] PATH_INFO: `/v3/projects/387be0c0c22c4808b6d23383ef2f4a9e/users/c911426639f04d4d823bd5e5155a70ae/roles/7b549d58d1724395b343609d895c80d0` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:23:31.206584 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:23:31.213773 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f604d89-9e7d-40ef-8c07-80e6abd0eda5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=387be0c0c22c4808b6d23383ef2f4a9e, user_id=c911426639f04d4d823bd5e5155a70ae, role_id=7b549d58d1724395b343609d895c80d0)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:23:31.216613 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f604d89-9e7d-40ef-8c07-80e6abd0eda5 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:23:31.235646 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1168/2357] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jul 14 09:23:31 2022] PUT /identity/v3/projects/387be0c0c22c4808b6d23383ef2f4a9e/users/c911426639f04d4d823bd5e5155a70ae/roles/7b549d58d1724395b343609d895c80d0 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:23:31.239674 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0549aac-9860-4ec1-9900-c7c2de0101a3 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:23:31.249951 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0549aac-9860-4ec1-9900-c7c2de0101a3 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:23:31.309778 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a32198-5818-4d41-8664-2bcc72c4e764 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:23:31.310502 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a32198-5818-4d41-8664-2bcc72c4e764 admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:23:31.310729 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a32198-5818-4d41-8664-2bcc72c4e764 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:23:31.311007 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a32198-5818-4d41-8664-2bcc72c4e764 admin admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:23:31.311779 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a32198-5818-4d41-8664-2bcc72c4e764 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:23:31.313386 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a32198-5818-4d41-8664-2bcc72c4e764 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:23:31.328448 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f604d89-9e7d-40ef-8c07-80e6abd0eda5 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:23:31.330370 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1190/2358] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jul 14 09:23:31 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 09:23:31.339565 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f604d89-9e7d-40ef-8c07-80e6abd0eda5 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:23:31.344462 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcca458-0a25-4abc-a00f-e475f6f9c642 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:23:31.345365 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcca458-0a25-4abc-a00f-e475f6f9c642 admin admin] REQUEST_METHOD: `PUT` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:23:31.345651 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcca458-0a25-4abc-a00f-e475f6f9c642 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:23:31.345930 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcca458-0a25-4abc-a00f-e475f6f9c642 admin admin] PATH_INFO: `/v3/projects/387be0c0c22c4808b6d23383ef2f4a9e/users/c911426639f04d4d823bd5e5155a70ae/roles/7b549d58d1724395b343609d895c80d0` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:23:31.348038 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbcca458-0a25-4abc-a00f-e475f6f9c642 admin admin] RBAC: Authorizing `identity:create_grant(project_id=387be0c0c22c4808b6d23383ef2f4a9e, user_id=c911426639f04d4d823bd5e5155a70ae, role_id=7b549d58d1724395b343609d895c80d0)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:23:31.351163 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbcca458-0a25-4abc-a00f-e475f6f9c642 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:23:31.365602 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1169/2359] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jul 14 09:23:31 2022] PUT /identity/v3/projects/387be0c0c22c4808b6d23383ef2f4a9e/users/c911426639f04d4d823bd5e5155a70ae/roles/7b549d58d1724395b343609d895c80d0 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 09:23:31.376247 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb98c89-cbb2-40b6-8870-d512387378b8 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:23:31.376664 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb98c89-cbb2-40b6-8870-d512387378b8 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:23:31.377082 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb98c89-cbb2-40b6-8870-d512387378b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:23:31.408166 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-5eb98c89-cbb2-40b6-8870-d512387378b8 None None] MFA Rules not processed for user `c911426639f04d4d823bd5e5155a70ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:23:31.409376 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-5eb98c89-cbb2-40b6-8870-d512387378b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:23:31.483335 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1191/2360] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:23:31 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:23:31.494782 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcca458-0a25-4abc-a00f-e475f6f9c642 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:23:31.496522 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-cbcca458-0a25-4abc-a00f-e475f6f9c642 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:23:31.558233 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcca458-0a25-4abc-a00f-e475f6f9c642 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:23:31.562244 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5eb89f4-c70b-4094-9eff-62b2aa506c60 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e76f1952a5b146808844b9f2b7395035', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:23:31.562901 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5eb89f4-c70b-4094-9eff-62b2aa506c60 service cinder] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:23:31.563211 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5eb89f4-c70b-4094-9eff-62b2aa506c60 service cinder] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:23:31.563603 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5eb89f4-c70b-4094-9eff-62b2aa506c60 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:23:31.574003 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5eb89f4-c70b-4094-9eff-62b2aa506c60 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:23:31.576449 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5eb89f4-c70b-4094-9eff-62b2aa506c60 service cinder] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:23:31.590577 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1170/2361] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:23:31 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:23:31.638219 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb98c89-cbb2-40b6-8870-d512387378b8 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:23:31.640190 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-5eb98c89-cbb2-40b6-8870-d512387378b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:23:31.653571 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb98c89-cbb2-40b6-8870-d512387378b8 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:23:31.657324 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fd38a6-1b8b-401b-9276-7fba680d066b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e76f1952a5b146808844b9f2b7395035', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:23:31.658020 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fd38a6-1b8b-401b-9276-7fba680d066b service cinder] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:23:31.658259 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fd38a6-1b8b-401b-9276-7fba680d066b service cinder] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:23:31.658536 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fd38a6-1b8b-401b-9276-7fba680d066b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:23:31.668877 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:23:31.721620 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fd38a6-1b8b-401b-9276-7fba680d066b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:23:31.723179 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fd38a6-1b8b-401b-9276-7fba680d066b service cinder] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:23:31.733105 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1192/2362] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:23:31 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:23:34.622036 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5eb89f4-c70b-4094-9eff-62b2aa506c60 service cinder] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:23:34.691149 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5eb89f4-c70b-4094-9eff-62b2aa506c60 service cinder] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:23:34.697251 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3a9178-4c5c-4d8e-8ca4-da933ea66974 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:23:34.698133 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3a9178-4c5c-4d8e-8ca4-da933ea66974 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:23:34.698420 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3a9178-4c5c-4d8e-8ca4-da933ea66974 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:23:34.699496 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3a9178-4c5c-4d8e-8ca4-da933ea66974 admin admin] PATH_INFO: `/v3/users/f0d6e1639747400899eaa8e4e03ce39c` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:23:34.700830 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3a9178-4c5c-4d8e-8ca4-da933ea66974 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f0d6e1639747400899eaa8e4e03ce39c)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:23:34.702869 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3a9178-4c5c-4d8e-8ca4-da933ea66974 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:23:34.754287 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-5e3a9178-4c5c-4d8e-8ca4-da933ea66974 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f0d6e1639747400899eaa8e4e03ce39c'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:23:34.762609 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-5e3a9178-4c5c-4d8e-8ca4-da933ea66974 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f0d6e1639747400899eaa8e4e03ce39c'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:23:34.763373 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-5e3a9178-4c5c-4d8e-8ca4-da933ea66974 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f0d6e1639747400899eaa8e4e03ce39c'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:23:34.764093 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-5e3a9178-4c5c-4d8e-8ca4-da933ea66974 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f0d6e1639747400899eaa8e4e03ce39c'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:23:34.772158 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-5e3a9178-4c5c-4d8e-8ca4-da933ea66974 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f0d6e1639747400899eaa8e4e03ce39c'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:23:34.786279 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:23:34.792326 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1171/2363] 10.222.0.26 () {58 vars in 1223 bytes} [Thu Jul 14 09:23:34 2022] DELETE /identity/v3/users/f0d6e1639747400899eaa8e4e03ce39c => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:23:34.797328 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e15e0d-59e4-43bc-a6e4-ca7d5ad83e70 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:23:34.797829 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e15e0d-59e4-43bc-a6e4-ca7d5ad83e70 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:23:34.797956 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e15e0d-59e4-43bc-a6e4-ca7d5ad83e70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:23:34.816084 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-80e15e0d-59e4-43bc-a6e4-ca7d5ad83e70 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:23:34.817276 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-80e15e0d-59e4-43bc-a6e4-ca7d5ad83e70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:23:34.876816 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1193/2364] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:23:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:23:34.886976 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3a9178-4c5c-4d8e-8ca4-da933ea66974 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:23:34.888587 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-5e3a9178-4c5c-4d8e-8ca4-da933ea66974 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:23:34.951145 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3a9178-4c5c-4d8e-8ca4-da933ea66974 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:23:34.955529 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220438dc-2fb9-45b1-a64c-c8bdc78d3da9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '952911c492444ffea4b211a24b564117', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:23:34.956177 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220438dc-2fb9-45b1-a64c-c8bdc78d3da9 service neutron] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:23:34.956459 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220438dc-2fb9-45b1-a64c-c8bdc78d3da9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:23:34.956731 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220438dc-2fb9-45b1-a64c-c8bdc78d3da9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:23:34.966428 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220438dc-2fb9-45b1-a64c-c8bdc78d3da9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:23:34.967902 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220438dc-2fb9-45b1-a64c-c8bdc78d3da9 service neutron] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:23:34.977516 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1172/2365] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:23:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:23:35.260007 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e15e0d-59e4-43bc-a6e4-ca7d5ad83e70 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:23:35.261463 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-80e15e0d-59e4-43bc-a6e4-ca7d5ad83e70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:23:35.274019 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e15e0d-59e4-43bc-a6e4-ca7d5ad83e70 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:23:35.279030 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e06444b-6fbe-40f2-a517-434effd75f93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:23:35.280019 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e06444b-6fbe-40f2-a517-434effd75f93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:23:35.280439 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e06444b-6fbe-40f2-a517-434effd75f93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:23:35.280841 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e06444b-6fbe-40f2-a517-434effd75f93 admin admin] PATH_INFO: `/v3/projects/e33c6a5d82dd4ba89a58a86e219b66c5` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:23:35.282088 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e06444b-6fbe-40f2-a517-434effd75f93 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e33c6a5d82dd4ba89a58a86e219b66c5)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:23:35.284035 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e06444b-6fbe-40f2-a517-434effd75f93 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:23:35.342789 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-9e06444b-6fbe-40f2-a517-434effd75f93 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e33c6a5d82dd4ba89a58a86e219b66c5', 'request_id': 'req-9e06444b-6fbe-40f2-a517-434effd75f93'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:23:35.349179 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-9e06444b-6fbe-40f2-a517-434effd75f93 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e33c6a5d82dd4ba89a58a86e219b66c5', 'request_id': 'req-9e06444b-6fbe-40f2-a517-434effd75f93'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:23:35.355906 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-9e06444b-6fbe-40f2-a517-434effd75f93 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e33c6a5d82dd4ba89a58a86e219b66c5', 'request_id': 'req-9e06444b-6fbe-40f2-a517-434effd75f93'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:23:35.375627 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-9e06444b-6fbe-40f2-a517-434effd75f93 admin admin] The token cache is being invalidate because project e33c6a5d82dd4ba89a58a86e219b66c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80382) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 09:23:35.375915 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-9e06444b-6fbe-40f2-a517-434effd75f93 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:23:35.376516 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-9e06444b-6fbe-40f2-a517-434effd75f93 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:23:35.378086 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1194/2366] 10.222.0.26 () {58 vars in 1232 bytes} [Thu Jul 14 09:23:35 2022] DELETE /identity/v3/projects/e33c6a5d82dd4ba89a58a86e219b66c5 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 09:23:35.382210 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220438dc-2fb9-45b1-a64c-c8bdc78d3da9 service neutron] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:23:35.391847 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220438dc-2fb9-45b1-a64c-c8bdc78d3da9 service neutron] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:23:35.447776 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e32e66-ad9b-42b5-bf80-e44c1a033045 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:23:35.448644 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e32e66-ad9b-42b5-bf80-e44c1a033045 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:23:35.448920 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e32e66-ad9b-42b5-bf80-e44c1a033045 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:23:35.449195 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e32e66-ad9b-42b5-bf80-e44c1a033045 admin admin] PATH_INFO: `/v3/users/06400fbd4bd94ee58be6c457ec6e789a` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:23:35.450393 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e32e66-ad9b-42b5-bf80-e44c1a033045 admin admin] RBAC: Authorizing `identity:delete_user(user_id=06400fbd4bd94ee58be6c457ec6e789a)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:23:35.452345 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e32e66-ad9b-42b5-bf80-e44c1a033045 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:23:35.501255 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-d7e32e66-ad9b-42b5-bf80-e44c1a033045 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '06400fbd4bd94ee58be6c457ec6e789a'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:23:35.511276 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-d7e32e66-ad9b-42b5-bf80-e44c1a033045 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '06400fbd4bd94ee58be6c457ec6e789a'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:23:35.511937 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-d7e32e66-ad9b-42b5-bf80-e44c1a033045 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '06400fbd4bd94ee58be6c457ec6e789a'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:23:35.512544 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-d7e32e66-ad9b-42b5-bf80-e44c1a033045 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '06400fbd4bd94ee58be6c457ec6e789a'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:23:35.519513 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-d7e32e66-ad9b-42b5-bf80-e44c1a033045 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '06400fbd4bd94ee58be6c457ec6e789a'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:23:35.538918 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1173/2367] 10.222.0.26 () {58 vars in 1223 bytes} [Thu Jul 14 09:23:35 2022] DELETE /identity/v3/users/06400fbd4bd94ee58be6c457ec6e789a => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:23:35.823479 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e06444b-6fbe-40f2-a517-434effd75f93 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:23:35.824971 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-9e06444b-6fbe-40f2-a517-434effd75f93 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:23:35.840010 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e06444b-6fbe-40f2-a517-434effd75f93 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:23:35.875138 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:23:35.900057 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c648732-a4a1-4dc3-a0b6-bc510a408668 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:23:35.900936 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c648732-a4a1-4dc3-a0b6-bc510a408668 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:23:35.901216 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c648732-a4a1-4dc3-a0b6-bc510a408668 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:23:35.901488 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c648732-a4a1-4dc3-a0b6-bc510a408668 admin admin] PATH_INFO: `/v3/projects/b02fa012cca342278d7a5330a4dc012f` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:23:35.902834 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c648732-a4a1-4dc3-a0b6-bc510a408668 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b02fa012cca342278d7a5330a4dc012f)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:23:35.904819 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c648732-a4a1-4dc3-a0b6-bc510a408668 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:23:35.954820 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-7c648732-a4a1-4dc3-a0b6-bc510a408668 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b02fa012cca342278d7a5330a4dc012f', 'request_id': 'req-7c648732-a4a1-4dc3-a0b6-bc510a408668'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:23:35.961373 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-7c648732-a4a1-4dc3-a0b6-bc510a408668 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b02fa012cca342278d7a5330a4dc012f', 'request_id': 'req-7c648732-a4a1-4dc3-a0b6-bc510a408668'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:23:35.967179 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-7c648732-a4a1-4dc3-a0b6-bc510a408668 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b02fa012cca342278d7a5330a4dc012f', 'request_id': 'req-7c648732-a4a1-4dc3-a0b6-bc510a408668'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:23:35.981325 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-7c648732-a4a1-4dc3-a0b6-bc510a408668 admin admin] The token cache is being invalidate because project b02fa012cca342278d7a5330a4dc012f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80382) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 09:23:35.981621 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-7c648732-a4a1-4dc3-a0b6-bc510a408668 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:23:35.982339 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-7c648732-a4a1-4dc3-a0b6-bc510a408668 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:23:35.983991 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1195/2368] 10.222.0.26 () {58 vars in 1232 bytes} [Thu Jul 14 09:23:35 2022] DELETE /identity/v3/projects/b02fa012cca342278d7a5330a4dc012f => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:23:35.988345 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e32e66-ad9b-42b5-bf80-e44c1a033045 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:23:35.989803 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-d7e32e66-ad9b-42b5-bf80-e44c1a033045 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:23:36.003071 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e32e66-ad9b-42b5-bf80-e44c1a033045 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:23:36.020627 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:23:36.069878 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def790da-1961-4ee0-9e2a-1550cd535b2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:23:36.070840 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def790da-1961-4ee0-9e2a-1550cd535b2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:23:36.071117 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def790da-1961-4ee0-9e2a-1550cd535b2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:23:36.071452 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def790da-1961-4ee0-9e2a-1550cd535b2f admin admin] PATH_INFO: `/v3/users/c911426639f04d4d823bd5e5155a70ae` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:23:36.072796 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def790da-1961-4ee0-9e2a-1550cd535b2f admin admin] RBAC: Authorizing `identity:delete_user(user_id=c911426639f04d4d823bd5e5155a70ae)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:23:36.074718 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def790da-1961-4ee0-9e2a-1550cd535b2f admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:23:36.125249 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-def790da-1961-4ee0-9e2a-1550cd535b2f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c911426639f04d4d823bd5e5155a70ae'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:23:36.132755 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-def790da-1961-4ee0-9e2a-1550cd535b2f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c911426639f04d4d823bd5e5155a70ae'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:23:36.133517 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-def790da-1961-4ee0-9e2a-1550cd535b2f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c911426639f04d4d823bd5e5155a70ae'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:23:36.134070 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-def790da-1961-4ee0-9e2a-1550cd535b2f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c911426639f04d4d823bd5e5155a70ae'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:23:36.142525 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-def790da-1961-4ee0-9e2a-1550cd535b2f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c911426639f04d4d823bd5e5155a70ae'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:23:36.161805 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1174/2369] 10.222.0.26 () {58 vars in 1223 bytes} [Thu Jul 14 09:23:35 2022] DELETE /identity/v3/users/c911426639f04d4d823bd5e5155a70ae => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 09:23:36.447379 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c648732-a4a1-4dc3-a0b6-bc510a408668 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:23:36.448822 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-7c648732-a4a1-4dc3-a0b6-bc510a408668 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:23:36.472407 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c648732-a4a1-4dc3-a0b6-bc510a408668 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:23:36.536215 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82938036-17d8-44fb-9309-c04cc04670b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:23:36.537136 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82938036-17d8-44fb-9309-c04cc04670b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:23:36.537385 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82938036-17d8-44fb-9309-c04cc04670b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:23:36.537654 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82938036-17d8-44fb-9309-c04cc04670b7 admin admin] PATH_INFO: `/v3/projects/387be0c0c22c4808b6d23383ef2f4a9e` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:23:36.538830 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82938036-17d8-44fb-9309-c04cc04670b7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=387be0c0c22c4808b6d23383ef2f4a9e)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:23:36.540814 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82938036-17d8-44fb-9309-c04cc04670b7 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:23:36.593807 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-82938036-17d8-44fb-9309-c04cc04670b7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '387be0c0c22c4808b6d23383ef2f4a9e', 'request_id': 'req-82938036-17d8-44fb-9309-c04cc04670b7'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:23:36.599860 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-82938036-17d8-44fb-9309-c04cc04670b7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '387be0c0c22c4808b6d23383ef2f4a9e', 'request_id': 'req-82938036-17d8-44fb-9309-c04cc04670b7'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:23:36.605348 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-82938036-17d8-44fb-9309-c04cc04670b7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '387be0c0c22c4808b6d23383ef2f4a9e', 'request_id': 'req-82938036-17d8-44fb-9309-c04cc04670b7'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:23:36.609537 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:23:36.620009 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-82938036-17d8-44fb-9309-c04cc04670b7 admin admin] The token cache is being invalidate because project 387be0c0c22c4808b6d23383ef2f4a9e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80382) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 09:23:36.620365 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-82938036-17d8-44fb-9309-c04cc04670b7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:23:36.621243 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-82938036-17d8-44fb-9309-c04cc04670b7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:23:36.623376 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1196/2370] 10.222.0.26 () {58 vars in 1232 bytes} [Thu Jul 14 09:23:36 2022] DELETE /identity/v3/projects/387be0c0c22c4808b6d23383ef2f4a9e => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:23:36.629304 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc69bdc-f351-4496-b95a-a3d70e1fe654 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:23:36.629616 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc69bdc-f351-4496-b95a-a3d70e1fe654 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:23:36.629896 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc69bdc-f351-4496-b95a-a3d70e1fe654 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:23:36.648874 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-acc69bdc-f351-4496-b95a-a3d70e1fe654 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:23:36.649991 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-acc69bdc-f351-4496-b95a-a3d70e1fe654 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:23:36.711279 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1175/2371] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:23:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 09:23:36.720006 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c053cfe0-467b-45e9-b00e-7dfd06246a1e None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:23:36.720290 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c053cfe0-467b-45e9-b00e-7dfd06246a1e None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:23:36.720575 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c053cfe0-467b-45e9-b00e-7dfd06246a1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:23:36.740257 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-c053cfe0-467b-45e9-b00e-7dfd06246a1e None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:23:36.741450 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-c053cfe0-467b-45e9-b00e-7dfd06246a1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:23:36.750109 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1197/2372] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:23:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:23:36.754213 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc69bdc-f351-4496-b95a-a3d70e1fe654 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:23:36.764559 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc69bdc-f351-4496-b95a-a3d70e1fe654 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:23:36.769476 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fc561c-00c2-4328-b378-8afe8f40eaed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:23:36.770134 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fc561c-00c2-4328-b378-8afe8f40eaed admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:23:36.770418 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fc561c-00c2-4328-b378-8afe8f40eaed admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:23:36.770699 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fc561c-00c2-4328-b378-8afe8f40eaed admin admin] PATH_INFO: `/v3/domains` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:23:36.771793 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fc561c-00c2-4328-b378-8afe8f40eaed admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:23:36.772103 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fc561c-00c2-4328-b378-8afe8f40eaed admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:23:36.773722 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fc561c-00c2-4328-b378-8afe8f40eaed admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:23:36.786186 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1176/2373] 10.222.0.26 () {58 vars in 1152 bytes} [Thu Jul 14 09:23:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:23:36.791088 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c053cfe0-467b-45e9-b00e-7dfd06246a1e None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:23:36.800662 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c053cfe0-467b-45e9-b00e-7dfd06246a1e None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:23:36.806539 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59113dd4-3ecb-4b69-84ea-8b8db02460e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:23:36.807259 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59113dd4-3ecb-4b69-84ea-8b8db02460e3 admin admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:23:36.807553 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59113dd4-3ecb-4b69-84ea-8b8db02460e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:23:36.807824 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59113dd4-3ecb-4b69-84ea-8b8db02460e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:23:36.808526 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59113dd4-3ecb-4b69-84ea-8b8db02460e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:23:36.810457 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59113dd4-3ecb-4b69-84ea-8b8db02460e3 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:23:36.840663 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1198/2374] 10.222.0.26 () {60 vars in 1152 bytes} [Thu Jul 14 09:23:36 2022] POST /identity/v3/projects => generated 373 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 09:23:36.846579 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fc561c-00c2-4328-b378-8afe8f40eaed admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:23:36.858866 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fc561c-00c2-4328-b378-8afe8f40eaed admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:23:36.883771 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:23:36.918546 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281cc14d-9d32-4e1b-af74-e99d40f3a0d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:23:36.919144 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281cc14d-9d32-4e1b-af74-e99d40f3a0d4 admin admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:23:36.919456 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281cc14d-9d32-4e1b-af74-e99d40f3a0d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:23:36.919740 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281cc14d-9d32-4e1b-af74-e99d40f3a0d4 admin admin] PATH_INFO: `/v3/users` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:23:36.920425 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281cc14d-9d32-4e1b-af74-e99d40f3a0d4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:23:36.922357 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281cc14d-9d32-4e1b-af74-e99d40f3a0d4 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:23:36.943050 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1177/2375] 10.222.0.26 () {60 vars in 1143 bytes} [Thu Jul 14 09:23:36 2022] POST /identity/v3/users => generated 287 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 09:23:36.947228 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59113dd4-3ecb-4b69-84ea-8b8db02460e3 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:23:36.957786 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59113dd4-3ecb-4b69-84ea-8b8db02460e3 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:23:36.962837 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92378c0-0ef6-4320-a8f5-78fe060a097c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:23:36.963555 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92378c0-0ef6-4320-a8f5-78fe060a097c admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:23:36.963838 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92378c0-0ef6-4320-a8f5-78fe060a097c admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:23:36.964112 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92378c0-0ef6-4320-a8f5-78fe060a097c admin admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:23:36.964844 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92378c0-0ef6-4320-a8f5-78fe060a097c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:23:36.966446 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92378c0-0ef6-4320-a8f5-78fe060a097c admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:23:36.977278 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1199/2376] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jul 14 09:23:36 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 09:23:36.981512 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281cc14d-9d32-4e1b-af74-e99d40f3a0d4 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:23:36.992782 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281cc14d-9d32-4e1b-af74-e99d40f3a0d4 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:23:36.999566 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401faae2-a77f-45c9-ad59-cfa3bb225b30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:23:37.000563 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401faae2-a77f-45c9-ad59-cfa3bb225b30 admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:23:37.000944 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401faae2-a77f-45c9-ad59-cfa3bb225b30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:23:37.001356 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401faae2-a77f-45c9-ad59-cfa3bb225b30 admin admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:23:37.002693 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401faae2-a77f-45c9-ad59-cfa3bb225b30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:23:37.005304 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401faae2-a77f-45c9-ad59-cfa3bb225b30 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:23:37.016789 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1178/2377] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jul 14 09:23:36 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 09:23:37.021507 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92378c0-0ef6-4320-a8f5-78fe060a097c admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:23:37.033738 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92378c0-0ef6-4320-a8f5-78fe060a097c admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:23:37.039626 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0c1d8b-2ac3-4e38-87f3-8f79e18d2005 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:23:37.040869 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0c1d8b-2ac3-4e38-87f3-8f79e18d2005 admin admin] REQUEST_METHOD: `PUT` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:23:37.041154 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0c1d8b-2ac3-4e38-87f3-8f79e18d2005 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:23:37.041435 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0c1d8b-2ac3-4e38-87f3-8f79e18d2005 admin admin] PATH_INFO: `/v3/projects/3b37a300617e443184095e87887b8435/users/a292f085b16c4319b6cfdf8aa13bb8b1/roles/d1691322ce6a4edab7ce722edf0f2988` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:23:37.062156 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0c1d8b-2ac3-4e38-87f3-8f79e18d2005 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b37a300617e443184095e87887b8435, user_id=a292f085b16c4319b6cfdf8aa13bb8b1, role_id=d1691322ce6a4edab7ce722edf0f2988)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:23:37.066417 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0c1d8b-2ac3-4e38-87f3-8f79e18d2005 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:23:37.082414 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1200/2378] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jul 14 09:23:37 2022] PUT /identity/v3/projects/3b37a300617e443184095e87887b8435/users/a292f085b16c4319b6cfdf8aa13bb8b1/roles/d1691322ce6a4edab7ce722edf0f2988 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:23:37.094939 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c94156-dde3-4cd9-a23e-43e7bbaa7e2b None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:23:37.095228 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c94156-dde3-4cd9-a23e-43e7bbaa7e2b None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:23:37.095549 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c94156-dde3-4cd9-a23e-43e7bbaa7e2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:23:37.124542 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-e2c94156-dde3-4cd9-a23e-43e7bbaa7e2b None None] MFA Rules not processed for user `a292f085b16c4319b6cfdf8aa13bb8b1`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:23:37.125777 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-e2c94156-dde3-4cd9-a23e-43e7bbaa7e2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:23:37.195029 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1179/2379] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:23:37 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:23:37.209738 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0c1d8b-2ac3-4e38-87f3-8f79e18d2005 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:23:37.212030 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-6b0c1d8b-2ac3-4e38-87f3-8f79e18d2005 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:23:37.271850 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0c1d8b-2ac3-4e38-87f3-8f79e18d2005 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:23:37.276690 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b215a5-f073-4e48-92fb-588b849f6ffa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e76f1952a5b146808844b9f2b7395035', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:23:37.277330 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b215a5-f073-4e48-92fb-588b849f6ffa service cinder] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:23:37.277611 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b215a5-f073-4e48-92fb-588b849f6ffa service cinder] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:23:37.277882 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b215a5-f073-4e48-92fb-588b849f6ffa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:23:37.287798 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b215a5-f073-4e48-92fb-588b849f6ffa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:23:37.289341 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b215a5-f073-4e48-92fb-588b849f6ffa service cinder] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:23:37.299114 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1201/2380] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:23:37 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:23:37.321422 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c94156-dde3-4cd9-a23e-43e7bbaa7e2b None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:23:37.387226 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c94156-dde3-4cd9-a23e-43e7bbaa7e2b None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:23:37.392487 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42be61b-f121-42ad-97f2-f0126d654685 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:23:37.393350 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42be61b-f121-42ad-97f2-f0126d654685 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:23:37.393633 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42be61b-f121-42ad-97f2-f0126d654685 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:23:37.393907 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42be61b-f121-42ad-97f2-f0126d654685 admin admin] PATH_INFO: `/v3/users/a292f085b16c4319b6cfdf8aa13bb8b1` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:23:37.395142 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42be61b-f121-42ad-97f2-f0126d654685 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a292f085b16c4319b6cfdf8aa13bb8b1)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:23:37.397420 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42be61b-f121-42ad-97f2-f0126d654685 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:23:37.439515 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:23:37.449336 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-f42be61b-f121-42ad-97f2-f0126d654685 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a292f085b16c4319b6cfdf8aa13bb8b1'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:23:37.458987 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-f42be61b-f121-42ad-97f2-f0126d654685 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a292f085b16c4319b6cfdf8aa13bb8b1'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:23:37.459776 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-f42be61b-f121-42ad-97f2-f0126d654685 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a292f085b16c4319b6cfdf8aa13bb8b1'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:23:37.460494 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-f42be61b-f121-42ad-97f2-f0126d654685 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a292f085b16c4319b6cfdf8aa13bb8b1'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:23:37.467752 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-f42be61b-f121-42ad-97f2-f0126d654685 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a292f085b16c4319b6cfdf8aa13bb8b1'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:23:37.484775 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1180/2381] 10.222.0.26 () {58 vars in 1223 bytes} [Thu Jul 14 09:23:37 2022] DELETE /identity/v3/users/a292f085b16c4319b6cfdf8aa13bb8b1 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:23:37.489393 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c823c775-3c22-4d0e-8eb4-8c0e77ba51ca None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:23:37.489705 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c823c775-3c22-4d0e-8eb4-8c0e77ba51ca None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:23:37.490002 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c823c775-3c22-4d0e-8eb4-8c0e77ba51ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:23:37.508869 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-c823c775-3c22-4d0e-8eb4-8c0e77ba51ca None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:23:37.510108 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-c823c775-3c22-4d0e-8eb4-8c0e77ba51ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:23:37.572116 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1202/2382] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:23:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:23:37.582353 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42be61b-f121-42ad-97f2-f0126d654685 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:23:37.583977 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-f42be61b-f121-42ad-97f2-f0126d654685 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:23:37.644199 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42be61b-f121-42ad-97f2-f0126d654685 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:23:37.648791 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d47f52c-e47c-4d26-babb-636a2af13731 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '952911c492444ffea4b211a24b564117', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:23:37.649446 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d47f52c-e47c-4d26-babb-636a2af13731 service neutron] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:23:37.649734 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d47f52c-e47c-4d26-babb-636a2af13731 service neutron] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:23:37.649999 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d47f52c-e47c-4d26-babb-636a2af13731 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:23:37.659589 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d47f52c-e47c-4d26-babb-636a2af13731 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:23:37.661019 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d47f52c-e47c-4d26-babb-636a2af13731 service neutron] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:23:37.671323 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1181/2383] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:23:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:23:37.945244 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c823c775-3c22-4d0e-8eb4-8c0e77ba51ca None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:23:37.946804 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-c823c775-3c22-4d0e-8eb4-8c0e77ba51ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:23:37.959695 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c823c775-3c22-4d0e-8eb4-8c0e77ba51ca None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:23:37.965231 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d563afa6-ab81-40d8-ac6e-f0c2f0c9fb25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:23:37.966108 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d563afa6-ab81-40d8-ac6e-f0c2f0c9fb25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:23:37.966380 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d563afa6-ab81-40d8-ac6e-f0c2f0c9fb25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:23:37.966651 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d563afa6-ab81-40d8-ac6e-f0c2f0c9fb25 admin admin] PATH_INFO: `/v3/projects/3b37a300617e443184095e87887b8435` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:23:37.967881 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d563afa6-ab81-40d8-ac6e-f0c2f0c9fb25 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3b37a300617e443184095e87887b8435)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:23:37.969787 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d563afa6-ab81-40d8-ac6e-f0c2f0c9fb25 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:23:38.006572 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:23:38.021291 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-d563afa6-ab81-40d8-ac6e-f0c2f0c9fb25 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3b37a300617e443184095e87887b8435', 'request_id': 'req-d563afa6-ab81-40d8-ac6e-f0c2f0c9fb25'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:23:38.027504 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-d563afa6-ab81-40d8-ac6e-f0c2f0c9fb25 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3b37a300617e443184095e87887b8435', 'request_id': 'req-d563afa6-ab81-40d8-ac6e-f0c2f0c9fb25'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:23:38.033371 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-d563afa6-ab81-40d8-ac6e-f0c2f0c9fb25 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3b37a300617e443184095e87887b8435', 'request_id': 'req-d563afa6-ab81-40d8-ac6e-f0c2f0c9fb25'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:23:38.047715 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-d563afa6-ab81-40d8-ac6e-f0c2f0c9fb25 admin admin] The token cache is being invalidate because project 3b37a300617e443184095e87887b8435 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80382) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 09:23:38.047998 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-d563afa6-ab81-40d8-ac6e-f0c2f0c9fb25 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:23:38.048704 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-d563afa6-ab81-40d8-ac6e-f0c2f0c9fb25 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:23:38.050353 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1203/2384] 10.222.0.26 () {58 vars in 1232 bytes} [Thu Jul 14 09:23:37 2022] DELETE /identity/v3/projects/3b37a300617e443184095e87887b8435 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:23:38.055799 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2a6dba-7cb2-425b-9130-57ad8ec259a1 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:23:38.056308 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2a6dba-7cb2-425b-9130-57ad8ec259a1 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:23:38.056424 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2a6dba-7cb2-425b-9130-57ad8ec259a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:23:38.075477 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-ef2a6dba-7cb2-425b-9130-57ad8ec259a1 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:23:38.076705 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-ef2a6dba-7cb2-425b-9130-57ad8ec259a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:23:38.137731 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1182/2385] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:23:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:23:38.145821 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3a598d-899c-4ac0-9ea0-953c82b630ef None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:23:38.146429 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3a598d-899c-4ac0-9ea0-953c82b630ef None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:23:38.146707 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3a598d-899c-4ac0-9ea0-953c82b630ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:23:38.165029 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-5f3a598d-899c-4ac0-9ea0-953c82b630ef None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:23:38.166234 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-5f3a598d-899c-4ac0-9ea0-953c82b630ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:23:38.174351 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1204/2386] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:23:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:23:38.178577 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2a6dba-7cb2-425b-9130-57ad8ec259a1 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:23:38.188344 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2a6dba-7cb2-425b-9130-57ad8ec259a1 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:23:38.193133 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7715d3-b352-4812-a8ed-200b9d873800 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:23:38.193780 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7715d3-b352-4812-a8ed-200b9d873800 admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:23:38.194062 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7715d3-b352-4812-a8ed-200b9d873800 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:23:38.194334 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7715d3-b352-4812-a8ed-200b9d873800 admin admin] PATH_INFO: `/v3/domains` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:23:38.195059 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7715d3-b352-4812-a8ed-200b9d873800 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:23:38.195414 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7715d3-b352-4812-a8ed-200b9d873800 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:23:38.196982 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7715d3-b352-4812-a8ed-200b9d873800 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:23:38.208575 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1183/2387] 10.222.0.26 () {58 vars in 1152 bytes} [Thu Jul 14 09:23:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:23:38.213612 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3a598d-899c-4ac0-9ea0-953c82b630ef None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:23:38.228357 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3a598d-899c-4ac0-9ea0-953c82b630ef None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:23:38.235892 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca02b8c-3235-4daa-a25c-559409a71c18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:23:38.236821 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca02b8c-3235-4daa-a25c-559409a71c18 admin admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:23:38.237224 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca02b8c-3235-4daa-a25c-559409a71c18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:23:38.237637 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca02b8c-3235-4daa-a25c-559409a71c18 admin admin] PATH_INFO: `/v3/projects` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:23:38.238728 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca02b8c-3235-4daa-a25c-559409a71c18 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:23:38.241864 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca02b8c-3235-4daa-a25c-559409a71c18 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:23:38.270405 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1205/2388] 10.222.0.26 () {60 vars in 1152 bytes} [Thu Jul 14 09:23:38 2022] POST /identity/v3/projects => generated 375 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 09:23:38.275103 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7715d3-b352-4812-a8ed-200b9d873800 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:23:38.285746 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7715d3-b352-4812-a8ed-200b9d873800 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:23:38.330372 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:23:38.347818 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361c9c6a-a251-40fd-8ac4-747a07eba09a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:23:38.348488 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361c9c6a-a251-40fd-8ac4-747a07eba09a admin admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:23:38.348767 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361c9c6a-a251-40fd-8ac4-747a07eba09a admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:23:38.349068 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361c9c6a-a251-40fd-8ac4-747a07eba09a admin admin] PATH_INFO: `/v3/users` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:23:38.349750 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361c9c6a-a251-40fd-8ac4-747a07eba09a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:23:38.351779 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361c9c6a-a251-40fd-8ac4-747a07eba09a admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:23:38.373289 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1184/2389] 10.222.0.26 () {60 vars in 1143 bytes} [Thu Jul 14 09:23:38 2022] POST /identity/v3/users => generated 288 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 09:23:38.377760 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca02b8c-3235-4daa-a25c-559409a71c18 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:23:38.388437 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca02b8c-3235-4daa-a25c-559409a71c18 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:23:38.393653 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6976df-8433-4856-b017-ad6e8d35e72f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:23:38.394312 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6976df-8433-4856-b017-ad6e8d35e72f admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:23:38.394589 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6976df-8433-4856-b017-ad6e8d35e72f admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:23:38.394865 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6976df-8433-4856-b017-ad6e8d35e72f admin admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:23:38.395725 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6976df-8433-4856-b017-ad6e8d35e72f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:23:38.397740 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6976df-8433-4856-b017-ad6e8d35e72f admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:23:38.408664 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1206/2390] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jul 14 09:23:38 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 09:23:38.412853 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361c9c6a-a251-40fd-8ac4-747a07eba09a admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:23:38.423954 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361c9c6a-a251-40fd-8ac4-747a07eba09a admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:23:38.429755 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bf1077-321f-4a4d-ae26-e2822c5323dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:23:38.430498 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bf1077-321f-4a4d-ae26-e2822c5323dc admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:23:38.430822 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bf1077-321f-4a4d-ae26-e2822c5323dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:23:38.431151 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bf1077-321f-4a4d-ae26-e2822c5323dc admin admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:23:38.431881 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38bf1077-321f-4a4d-ae26-e2822c5323dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:23:38.433499 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38bf1077-321f-4a4d-ae26-e2822c5323dc admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:23:38.444202 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1185/2391] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jul 14 09:23:38 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 09:23:38.448444 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6976df-8433-4856-b017-ad6e8d35e72f admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:23:38.458676 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6976df-8433-4856-b017-ad6e8d35e72f admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:23:38.464020 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dfb7d5-b206-47a6-86bb-75b789d33f0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:23:38.464843 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dfb7d5-b206-47a6-86bb-75b789d33f0b admin admin] REQUEST_METHOD: `PUT` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:23:38.465129 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dfb7d5-b206-47a6-86bb-75b789d33f0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:23:38.465410 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dfb7d5-b206-47a6-86bb-75b789d33f0b admin admin] PATH_INFO: `/v3/projects/de934895e37042eaa074fa024884609f/users/222faef7dedf4034b120eba111bebac6/roles/d1691322ce6a4edab7ce722edf0f2988` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:23:38.481292 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3dfb7d5-b206-47a6-86bb-75b789d33f0b admin admin] RBAC: Authorizing `identity:create_grant(project_id=de934895e37042eaa074fa024884609f, user_id=222faef7dedf4034b120eba111bebac6, role_id=d1691322ce6a4edab7ce722edf0f2988)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:23:38.484010 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3dfb7d5-b206-47a6-86bb-75b789d33f0b admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:23:38.500198 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1207/2392] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jul 14 09:23:38 2022] PUT /identity/v3/projects/de934895e37042eaa074fa024884609f/users/222faef7dedf4034b120eba111bebac6/roles/d1691322ce6a4edab7ce722edf0f2988 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:23:38.510641 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d274e6a5-d328-413b-9f1f-5c1e627dd1d7 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:23:38.510951 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d274e6a5-d328-413b-9f1f-5c1e627dd1d7 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:23:38.511224 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d274e6a5-d328-413b-9f1f-5c1e627dd1d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:23:38.539728 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-d274e6a5-d328-413b-9f1f-5c1e627dd1d7 None None] MFA Rules not processed for user `222faef7dedf4034b120eba111bebac6`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:23:38.540950 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-d274e6a5-d328-413b-9f1f-5c1e627dd1d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:23:38.610720 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1186/2393] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:23:38 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 09:23:38.615502 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dfb7d5-b206-47a6-86bb-75b789d33f0b admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:23:38.679982 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dfb7d5-b206-47a6-86bb-75b789d33f0b admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:23:38.685315 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8736da-4585-4939-9b7d-9cf28f1ca567 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:23:38.685982 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8736da-4585-4939-9b7d-9cf28f1ca567 admin admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:23:38.686267 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8736da-4585-4939-9b7d-9cf28f1ca567 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:23:38.686541 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8736da-4585-4939-9b7d-9cf28f1ca567 admin admin] PATH_INFO: `/v3/projects` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:23:38.687236 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8736da-4585-4939-9b7d-9cf28f1ca567 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:23:38.689212 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8736da-4585-4939-9b7d-9cf28f1ca567 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:23:38.717069 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1208/2394] 10.222.0.26 () {60 vars in 1152 bytes} [Thu Jul 14 09:23:38 2022] POST /identity/v3/projects => generated 373 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 09:23:38.721266 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d274e6a5-d328-413b-9f1f-5c1e627dd1d7 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:23:38.732402 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d274e6a5-d328-413b-9f1f-5c1e627dd1d7 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:23:38.791098 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e917bc-f213-4d93-812a-1c44302c4046 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:23:38.791818 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e917bc-f213-4d93-812a-1c44302c4046 admin admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:23:38.792104 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e917bc-f213-4d93-812a-1c44302c4046 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:23:38.792379 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e917bc-f213-4d93-812a-1c44302c4046 admin admin] PATH_INFO: `/v3/users` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:23:38.793082 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e917bc-f213-4d93-812a-1c44302c4046 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:23:38.795004 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e917bc-f213-4d93-812a-1c44302c4046 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:23:38.818825 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1187/2395] 10.222.0.26 () {60 vars in 1143 bytes} [Thu Jul 14 09:23:38 2022] POST /identity/v3/users => generated 287 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 09:23:38.820295 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8736da-4585-4939-9b7d-9cf28f1ca567 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:23:38.830990 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8736da-4585-4939-9b7d-9cf28f1ca567 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:23:38.836015 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c50dd3f-6aab-4606-a7e8-42aa0e2f6a1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:23:38.836883 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c50dd3f-6aab-4606-a7e8-42aa0e2f6a1b admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:23:38.837216 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c50dd3f-6aab-4606-a7e8-42aa0e2f6a1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:23:38.837486 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c50dd3f-6aab-4606-a7e8-42aa0e2f6a1b admin admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:23:38.838232 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c50dd3f-6aab-4606-a7e8-42aa0e2f6a1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:23:38.839906 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c50dd3f-6aab-4606-a7e8-42aa0e2f6a1b admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:23:38.851226 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1209/2396] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jul 14 09:23:38 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 09:23:38.855522 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e917bc-f213-4d93-812a-1c44302c4046 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:23:38.865482 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e917bc-f213-4d93-812a-1c44302c4046 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:23:38.870425 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa50c38a-498b-4fc8-8c37-15312235dbe9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:23:38.871095 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa50c38a-498b-4fc8-8c37-15312235dbe9 admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:23:38.871412 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa50c38a-498b-4fc8-8c37-15312235dbe9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:23:38.871693 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa50c38a-498b-4fc8-8c37-15312235dbe9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:23:38.872455 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa50c38a-498b-4fc8-8c37-15312235dbe9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:23:38.874102 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa50c38a-498b-4fc8-8c37-15312235dbe9 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:23:38.884575 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1188/2397] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jul 14 09:23:38 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 09:23:38.888888 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c50dd3f-6aab-4606-a7e8-42aa0e2f6a1b admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:23:38.899355 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c50dd3f-6aab-4606-a7e8-42aa0e2f6a1b admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:23:38.904112 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41041934-13bb-4083-beed-a6dc7620a970 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:23:38.905047 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41041934-13bb-4083-beed-a6dc7620a970 admin admin] REQUEST_METHOD: `PUT` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:23:38.905329 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41041934-13bb-4083-beed-a6dc7620a970 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:23:38.905600 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41041934-13bb-4083-beed-a6dc7620a970 admin admin] PATH_INFO: `/v3/projects/de916ae223bc48b6a426100fb5a4ac3c/users/8dcb7f8377cb4b2997aa07b47f9b8e4c/roles/d1691322ce6a4edab7ce722edf0f2988` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:23:38.920424 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41041934-13bb-4083-beed-a6dc7620a970 admin admin] RBAC: Authorizing `identity:create_grant(project_id=de916ae223bc48b6a426100fb5a4ac3c, user_id=8dcb7f8377cb4b2997aa07b47f9b8e4c, role_id=d1691322ce6a4edab7ce722edf0f2988)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:23:38.923094 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41041934-13bb-4083-beed-a6dc7620a970 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:23:38.937734 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1210/2398] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jul 14 09:23:38 2022] PUT /identity/v3/projects/de916ae223bc48b6a426100fb5a4ac3c/users/8dcb7f8377cb4b2997aa07b47f9b8e4c/roles/d1691322ce6a4edab7ce722edf0f2988 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:23:39.003163 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfda57b4-4a66-4051-b039-dbdf3e149324 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:23:39.003512 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfda57b4-4a66-4051-b039-dbdf3e149324 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:23:39.003848 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfda57b4-4a66-4051-b039-dbdf3e149324 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:23:39.034094 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-dfda57b4-4a66-4051-b039-dbdf3e149324 None None] MFA Rules not processed for user `8dcb7f8377cb4b2997aa07b47f9b8e4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:23:39.035332 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-dfda57b4-4a66-4051-b039-dbdf3e149324 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:23:39.057758 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:23:39.105628 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1189/2399] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:23:39 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:23:39.110651 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41041934-13bb-4083-beed-a6dc7620a970 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:23:39.117092 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:23:39.177559 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41041934-13bb-4083-beed-a6dc7620a970 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:23:39.182884 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8909ccc1-13ee-4db4-8ebf-29da7c63522e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:23:39.183586 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8909ccc1-13ee-4db4-8ebf-29da7c63522e admin admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:23:39.183864 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8909ccc1-13ee-4db4-8ebf-29da7c63522e admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:23:39.184138 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8909ccc1-13ee-4db4-8ebf-29da7c63522e admin admin] PATH_INFO: `/v3/projects` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:23:39.184833 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8909ccc1-13ee-4db4-8ebf-29da7c63522e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:23:39.186748 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8909ccc1-13ee-4db4-8ebf-29da7c63522e admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:23:39.215489 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1211/2400] 10.222.0.26 () {60 vars in 1152 bytes} [Thu Jul 14 09:23:39 2022] POST /identity/v3/projects => generated 377 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 09:23:39.221515 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfda57b4-4a66-4051-b039-dbdf3e149324 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:23:39.235223 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfda57b4-4a66-4051-b039-dbdf3e149324 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:23:39.298725 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f057a2-9f0f-4359-a8a8-b323ad8a3f0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:23:39.299349 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f057a2-9f0f-4359-a8a8-b323ad8a3f0a admin admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:23:39.299636 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f057a2-9f0f-4359-a8a8-b323ad8a3f0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:23:39.299910 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f057a2-9f0f-4359-a8a8-b323ad8a3f0a admin admin] PATH_INFO: `/v3/users` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:23:39.300636 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f057a2-9f0f-4359-a8a8-b323ad8a3f0a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:23:39.302587 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f057a2-9f0f-4359-a8a8-b323ad8a3f0a admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:23:39.325145 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1190/2401] 10.222.0.26 () {60 vars in 1143 bytes} [Thu Jul 14 09:23:39 2022] POST /identity/v3/users => generated 295 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 09:23:39.328827 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8909ccc1-13ee-4db4-8ebf-29da7c63522e admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:23:39.340604 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8909ccc1-13ee-4db4-8ebf-29da7c63522e admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:23:39.346532 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1d732f-6368-427c-b887-dcc538649052 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:23:39.347463 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1d732f-6368-427c-b887-dcc538649052 admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:23:39.347620 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1d732f-6368-427c-b887-dcc538649052 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:23:39.347874 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1d732f-6368-427c-b887-dcc538649052 admin admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:23:39.348624 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1d732f-6368-427c-b887-dcc538649052 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:23:39.350240 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1d732f-6368-427c-b887-dcc538649052 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:23:39.362401 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1212/2402] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jul 14 09:23:39 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 09:23:39.366605 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f057a2-9f0f-4359-a8a8-b323ad8a3f0a admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:23:39.379793 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f057a2-9f0f-4359-a8a8-b323ad8a3f0a admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:23:39.385940 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6810c86-b306-4000-aa39-6db9224f7bdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:23:39.386851 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6810c86-b306-4000-aa39-6db9224f7bdc admin admin] REQUEST_METHOD: `PUT` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:23:39.387134 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6810c86-b306-4000-aa39-6db9224f7bdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:23:39.387443 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6810c86-b306-4000-aa39-6db9224f7bdc admin admin] PATH_INFO: `/v3/projects/7cdc767987664cada71b4f493fca8533/users/664a8f9f2fe04d99bc99946ec36f8b2b/roles/7b549d58d1724395b343609d895c80d0` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:23:39.404360 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6810c86-b306-4000-aa39-6db9224f7bdc admin admin] RBAC: Authorizing `identity:create_grant(project_id=7cdc767987664cada71b4f493fca8533, user_id=664a8f9f2fe04d99bc99946ec36f8b2b, role_id=7b549d58d1724395b343609d895c80d0)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:23:39.407212 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6810c86-b306-4000-aa39-6db9224f7bdc admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:23:39.420930 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1191/2403] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jul 14 09:23:39 2022] PUT /identity/v3/projects/7cdc767987664cada71b4f493fca8533/users/664a8f9f2fe04d99bc99946ec36f8b2b/roles/7b549d58d1724395b343609d895c80d0 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:23:39.425377 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1d732f-6368-427c-b887-dcc538649052 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:23:39.436467 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1d732f-6368-427c-b887-dcc538649052 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:23:39.497554 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084e8df0-16cb-40a1-ac91-df940841e459 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:23:39.498146 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084e8df0-16cb-40a1-ac91-df940841e459 admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:23:39.498430 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084e8df0-16cb-40a1-ac91-df940841e459 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:23:39.498714 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084e8df0-16cb-40a1-ac91-df940841e459 admin admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:23:39.499481 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084e8df0-16cb-40a1-ac91-df940841e459 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:23:39.501480 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084e8df0-16cb-40a1-ac91-df940841e459 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:23:39.512118 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1213/2404] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jul 14 09:23:39 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 09:23:39.516319 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6810c86-b306-4000-aa39-6db9224f7bdc admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:23:39.527232 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6810c86-b306-4000-aa39-6db9224f7bdc admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:23:39.532121 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3102378-2c6c-4fb6-aa50-c10fb36da1d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:23:39.533027 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3102378-2c6c-4fb6-aa50-c10fb36da1d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:23:39.533306 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3102378-2c6c-4fb6-aa50-c10fb36da1d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:23:39.533579 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3102378-2c6c-4fb6-aa50-c10fb36da1d4 admin admin] PATH_INFO: `/v3/projects/7cdc767987664cada71b4f493fca8533/users/664a8f9f2fe04d99bc99946ec36f8b2b/roles/7b549d58d1724395b343609d895c80d0` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:23:39.537826 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3102378-2c6c-4fb6-aa50-c10fb36da1d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7cdc767987664cada71b4f493fca8533, user_id=664a8f9f2fe04d99bc99946ec36f8b2b, role_id=7b549d58d1724395b343609d895c80d0)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:23:39.540623 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3102378-2c6c-4fb6-aa50-c10fb36da1d4 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:23:39.552116 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1192/2405] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jul 14 09:23:39 2022] PUT /identity/v3/projects/7cdc767987664cada71b4f493fca8533/users/664a8f9f2fe04d99bc99946ec36f8b2b/roles/7b549d58d1724395b343609d895c80d0 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:23:39.562852 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562fe137-a85e-44fa-b904-84d2cf2ea22e None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:23:39.563226 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562fe137-a85e-44fa-b904-84d2cf2ea22e None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:23:39.563506 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562fe137-a85e-44fa-b904-84d2cf2ea22e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:23:39.592002 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-562fe137-a85e-44fa-b904-84d2cf2ea22e None None] MFA Rules not processed for user `664a8f9f2fe04d99bc99946ec36f8b2b`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:23:39.593458 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-562fe137-a85e-44fa-b904-84d2cf2ea22e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:23:39.668221 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:23:39.672292 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1214/2406] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:23:39 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:23:39.684798 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3102378-2c6c-4fb6-aa50-c10fb36da1d4 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:23:39.686536 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-d3102378-2c6c-4fb6-aa50-c10fb36da1d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:23:39.747164 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3102378-2c6c-4fb6-aa50-c10fb36da1d4 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:23:39.751494 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bcc3e4-38ef-4047-9d56-15285caaa390 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e76f1952a5b146808844b9f2b7395035', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:23:39.752170 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bcc3e4-38ef-4047-9d56-15285caaa390 service cinder] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:23:39.752453 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bcc3e4-38ef-4047-9d56-15285caaa390 service cinder] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:23:39.752725 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bcc3e4-38ef-4047-9d56-15285caaa390 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:23:39.782127 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:23:39.814658 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bcc3e4-38ef-4047-9d56-15285caaa390 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:23:39.816053 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bcc3e4-38ef-4047-9d56-15285caaa390 service cinder] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:23:39.825408 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1193/2407] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:23:39 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 09:23:41.487376 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562fe137-a85e-44fa-b904-84d2cf2ea22e None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:23:41.488932 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-562fe137-a85e-44fa-b904-84d2cf2ea22e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:23:41.502094 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562fe137-a85e-44fa-b904-84d2cf2ea22e None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:23:41.506160 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5308928a-dea9-455f-9d2b-55f161b4c60e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e76f1952a5b146808844b9f2b7395035', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:23:41.506896 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5308928a-dea9-455f-9d2b-55f161b4c60e service cinder] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:23:41.507208 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5308928a-dea9-455f-9d2b-55f161b4c60e service cinder] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:23:41.507490 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5308928a-dea9-455f-9d2b-55f161b4c60e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:23:41.565371 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5308928a-dea9-455f-9d2b-55f161b4c60e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:23:41.566852 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5308928a-dea9-455f-9d2b-55f161b4c60e service cinder] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:23:41.575822 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1215/2408] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:23:41 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:23:42.123410 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bcc3e4-38ef-4047-9d56-15285caaa390 service cinder] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:23:42.130636 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bcc3e4-38ef-4047-9d56-15285caaa390 service cinder] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:23:42.134339 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a394e9-07fe-42fe-8706-96360bf6e77f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e76f1952a5b146808844b9f2b7395035', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:23:42.135028 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a394e9-07fe-42fe-8706-96360bf6e77f service cinder] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:23:42.135335 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a394e9-07fe-42fe-8706-96360bf6e77f service cinder] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:23:42.135612 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a394e9-07fe-42fe-8706-96360bf6e77f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:23:42.144993 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a394e9-07fe-42fe-8706-96360bf6e77f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:23:42.146735 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a394e9-07fe-42fe-8706-96360bf6e77f service cinder] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:23:42.156463 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1194/2409] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:23:42 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:23:46.047082 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5308928a-dea9-455f-9d2b-55f161b4c60e service cinder] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:23:46.118834 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5308928a-dea9-455f-9d2b-55f161b4c60e service cinder] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:23:46.123648 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5140382f-075e-4b40-ad0c-890677960a2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:23:46.124549 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5140382f-075e-4b40-ad0c-890677960a2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:23:46.124824 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5140382f-075e-4b40-ad0c-890677960a2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:23:46.125089 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5140382f-075e-4b40-ad0c-890677960a2a admin admin] PATH_INFO: `/v3/users/222faef7dedf4034b120eba111bebac6` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:23:46.126631 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5140382f-075e-4b40-ad0c-890677960a2a admin admin] RBAC: Authorizing `identity:delete_user(user_id=222faef7dedf4034b120eba111bebac6)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:23:46.128964 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5140382f-075e-4b40-ad0c-890677960a2a admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:23:46.180023 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-5140382f-075e-4b40-ad0c-890677960a2a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '222faef7dedf4034b120eba111bebac6'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:23:46.180678 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-5140382f-075e-4b40-ad0c-890677960a2a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '222faef7dedf4034b120eba111bebac6'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:23:46.192700 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-5140382f-075e-4b40-ad0c-890677960a2a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '222faef7dedf4034b120eba111bebac6'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:23:46.200987 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-5140382f-075e-4b40-ad0c-890677960a2a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '222faef7dedf4034b120eba111bebac6'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:23:46.201643 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-5140382f-075e-4b40-ad0c-890677960a2a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '222faef7dedf4034b120eba111bebac6'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:23:46.213797 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1216/2410] 10.222.0.26 () {58 vars in 1223 bytes} [Thu Jul 14 09:23:46 2022] DELETE /identity/v3/users/222faef7dedf4034b120eba111bebac6 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:23:46.218365 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3af1050-6f07-4a1b-85f5-fa0a156601ba None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:23:46.218829 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3af1050-6f07-4a1b-85f5-fa0a156601ba None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:23:46.218922 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3af1050-6f07-4a1b-85f5-fa0a156601ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:23:46.237062 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-a3af1050-6f07-4a1b-85f5-fa0a156601ba None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:23:46.238537 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-a3af1050-6f07-4a1b-85f5-fa0a156601ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:23:46.314547 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1195/2411] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:23:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:23:46.326319 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5140382f-075e-4b40-ad0c-890677960a2a admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:23:46.328010 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-5140382f-075e-4b40-ad0c-890677960a2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:23:46.350866 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:23:46.391563 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5140382f-075e-4b40-ad0c-890677960a2a admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:23:46.396031 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cad7ef-87d9-4c0e-8441-ff6492847fbf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '952911c492444ffea4b211a24b564117', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:23:46.396681 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cad7ef-87d9-4c0e-8441-ff6492847fbf service neutron] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:23:46.396959 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cad7ef-87d9-4c0e-8441-ff6492847fbf service neutron] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:23:46.397230 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cad7ef-87d9-4c0e-8441-ff6492847fbf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:23:46.407512 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5cad7ef-87d9-4c0e-8441-ff6492847fbf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:23:46.408947 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5cad7ef-87d9-4c0e-8441-ff6492847fbf service neutron] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:23:46.418676 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1217/2412] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:23:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:23:46.705201 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3af1050-6f07-4a1b-85f5-fa0a156601ba None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:23:46.706850 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-a3af1050-6f07-4a1b-85f5-fa0a156601ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:23:46.721946 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3af1050-6f07-4a1b-85f5-fa0a156601ba None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:23:46.728127 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d16dd9-7fd1-4270-925d-f5d252a717db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:23:46.729169 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d16dd9-7fd1-4270-925d-f5d252a717db admin admin] REQUEST_METHOD: `DELETE` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:23:46.729738 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d16dd9-7fd1-4270-925d-f5d252a717db admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:23:46.730116 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d16dd9-7fd1-4270-925d-f5d252a717db admin admin] PATH_INFO: `/v3/projects/de934895e37042eaa074fa024884609f` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:23:46.731733 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d16dd9-7fd1-4270-925d-f5d252a717db admin admin] RBAC: Authorizing `identity:delete_project(project_id=de934895e37042eaa074fa024884609f)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:23:46.733641 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d16dd9-7fd1-4270-925d-f5d252a717db admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:23:46.782664 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-92d16dd9-7fd1-4270-925d-f5d252a717db admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'de934895e37042eaa074fa024884609f', 'request_id': 'req-92d16dd9-7fd1-4270-925d-f5d252a717db'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:23:46.788975 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-92d16dd9-7fd1-4270-925d-f5d252a717db admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'de934895e37042eaa074fa024884609f', 'request_id': 'req-92d16dd9-7fd1-4270-925d-f5d252a717db'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:23:46.794820 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-92d16dd9-7fd1-4270-925d-f5d252a717db admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'de934895e37042eaa074fa024884609f', 'request_id': 'req-92d16dd9-7fd1-4270-925d-f5d252a717db'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:23:46.808387 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-92d16dd9-7fd1-4270-925d-f5d252a717db admin admin] The token cache is being invalidate because project de934895e37042eaa074fa024884609f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80381) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 09:23:46.808671 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-92d16dd9-7fd1-4270-925d-f5d252a717db admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:23:46.809392 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-92d16dd9-7fd1-4270-925d-f5d252a717db admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:23:46.811146 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1196/2413] 10.222.0.26 () {58 vars in 1232 bytes} [Thu Jul 14 09:23:46 2022] DELETE /identity/v3/projects/de934895e37042eaa074fa024884609f => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:23:46.815299 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cad7ef-87d9-4c0e-8441-ff6492847fbf service neutron] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:23:46.825117 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cad7ef-87d9-4c0e-8441-ff6492847fbf service neutron] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:23:46.887861 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8598933c-566e-40e5-aff3-ba837b48718b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:23:46.888666 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8598933c-566e-40e5-aff3-ba837b48718b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:23:46.888970 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8598933c-566e-40e5-aff3-ba837b48718b admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:23:46.889228 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8598933c-566e-40e5-aff3-ba837b48718b admin admin] PATH_INFO: `/v3/users/8dcb7f8377cb4b2997aa07b47f9b8e4c` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:23:46.890451 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8598933c-566e-40e5-aff3-ba837b48718b admin admin] RBAC: Authorizing `identity:delete_user(user_id=8dcb7f8377cb4b2997aa07b47f9b8e4c)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:23:46.892438 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8598933c-566e-40e5-aff3-ba837b48718b admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:23:46.940943 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-8598933c-566e-40e5-aff3-ba837b48718b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8dcb7f8377cb4b2997aa07b47f9b8e4c'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:23:46.941590 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-8598933c-566e-40e5-aff3-ba837b48718b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8dcb7f8377cb4b2997aa07b47f9b8e4c'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:23:46.953489 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-8598933c-566e-40e5-aff3-ba837b48718b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8dcb7f8377cb4b2997aa07b47f9b8e4c'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:23:46.962337 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-8598933c-566e-40e5-aff3-ba837b48718b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8dcb7f8377cb4b2997aa07b47f9b8e4c'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:23:46.962960 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-8598933c-566e-40e5-aff3-ba837b48718b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8dcb7f8377cb4b2997aa07b47f9b8e4c'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:23:46.975524 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1218/2414] 10.222.0.26 () {58 vars in 1223 bytes} [Thu Jul 14 09:23:46 2022] DELETE /identity/v3/users/8dcb7f8377cb4b2997aa07b47f9b8e4c => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:23:47.318769 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d16dd9-7fd1-4270-925d-f5d252a717db admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:23:47.320821 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-92d16dd9-7fd1-4270-925d-f5d252a717db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:23:47.336059 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d16dd9-7fd1-4270-925d-f5d252a717db admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:23:47.371918 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:23:47.403447 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eafddf0-5b60-4614-bd71-ac9bdfb97240 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:23:47.404161 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eafddf0-5b60-4614-bd71-ac9bdfb97240 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:23:47.404445 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eafddf0-5b60-4614-bd71-ac9bdfb97240 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:23:47.404722 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eafddf0-5b60-4614-bd71-ac9bdfb97240 admin admin] PATH_INFO: `/v3/projects/de916ae223bc48b6a426100fb5a4ac3c` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:23:47.406085 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eafddf0-5b60-4614-bd71-ac9bdfb97240 admin admin] RBAC: Authorizing `identity:delete_project(project_id=de916ae223bc48b6a426100fb5a4ac3c)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:23:47.408078 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eafddf0-5b60-4614-bd71-ac9bdfb97240 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:23:47.458116 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-5eafddf0-5b60-4614-bd71-ac9bdfb97240 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'de916ae223bc48b6a426100fb5a4ac3c', 'request_id': 'req-5eafddf0-5b60-4614-bd71-ac9bdfb97240'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:23:47.464360 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-5eafddf0-5b60-4614-bd71-ac9bdfb97240 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'de916ae223bc48b6a426100fb5a4ac3c', 'request_id': 'req-5eafddf0-5b60-4614-bd71-ac9bdfb97240'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:23:47.470145 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-5eafddf0-5b60-4614-bd71-ac9bdfb97240 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'de916ae223bc48b6a426100fb5a4ac3c', 'request_id': 'req-5eafddf0-5b60-4614-bd71-ac9bdfb97240'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:23:47.485589 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-5eafddf0-5b60-4614-bd71-ac9bdfb97240 admin admin] The token cache is being invalidate because project de916ae223bc48b6a426100fb5a4ac3c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80381) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 09:23:47.485905 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-5eafddf0-5b60-4614-bd71-ac9bdfb97240 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:23:47.486506 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-5eafddf0-5b60-4614-bd71-ac9bdfb97240 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:23:47.488570 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1197/2415] 10.222.0.26 () {58 vars in 1232 bytes} [Thu Jul 14 09:23:47 2022] DELETE /identity/v3/projects/de916ae223bc48b6a426100fb5a4ac3c => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:23:47.492663 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8598933c-566e-40e5-aff3-ba837b48718b admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:23:47.494205 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-8598933c-566e-40e5-aff3-ba837b48718b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:23:47.508875 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8598933c-566e-40e5-aff3-ba837b48718b admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:23:47.548359 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:23:47.569618 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b02fc8-ce4b-4ff9-a7db-462e1ced379c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:23:47.570527 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b02fc8-ce4b-4ff9-a7db-462e1ced379c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:23:47.570811 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b02fc8-ce4b-4ff9-a7db-462e1ced379c admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:23:47.571103 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b02fc8-ce4b-4ff9-a7db-462e1ced379c admin admin] PATH_INFO: `/v3/users/664a8f9f2fe04d99bc99946ec36f8b2b` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:23:47.572504 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b02fc8-ce4b-4ff9-a7db-462e1ced379c admin admin] RBAC: Authorizing `identity:delete_user(user_id=664a8f9f2fe04d99bc99946ec36f8b2b)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:23:47.574460 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b02fc8-ce4b-4ff9-a7db-462e1ced379c admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:23:47.632332 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-59b02fc8-ce4b-4ff9-a7db-462e1ced379c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '664a8f9f2fe04d99bc99946ec36f8b2b'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:23:47.633023 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-59b02fc8-ce4b-4ff9-a7db-462e1ced379c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '664a8f9f2fe04d99bc99946ec36f8b2b'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:23:47.644754 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-59b02fc8-ce4b-4ff9-a7db-462e1ced379c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '664a8f9f2fe04d99bc99946ec36f8b2b'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:23:47.653508 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-59b02fc8-ce4b-4ff9-a7db-462e1ced379c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '664a8f9f2fe04d99bc99946ec36f8b2b'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:23:47.653678 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-59b02fc8-ce4b-4ff9-a7db-462e1ced379c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '664a8f9f2fe04d99bc99946ec36f8b2b'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:23:47.667311 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1219/2416] 10.222.0.26 () {58 vars in 1223 bytes} [Thu Jul 14 09:23:47 2022] DELETE /identity/v3/users/664a8f9f2fe04d99bc99946ec36f8b2b => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:23:47.973872 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eafddf0-5b60-4614-bd71-ac9bdfb97240 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:23:47.975402 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-5eafddf0-5b60-4614-bd71-ac9bdfb97240 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:23:47.997701 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eafddf0-5b60-4614-bd71-ac9bdfb97240 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:23:48.064990 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efd06a8-debb-4ca4-a7ca-abac52ae6ca7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:23:48.065870 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efd06a8-debb-4ca4-a7ca-abac52ae6ca7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:23:48.066160 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efd06a8-debb-4ca4-a7ca-abac52ae6ca7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:23:48.066437 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efd06a8-debb-4ca4-a7ca-abac52ae6ca7 admin admin] PATH_INFO: `/v3/projects/7cdc767987664cada71b4f493fca8533` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:23:48.067680 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efd06a8-debb-4ca4-a7ca-abac52ae6ca7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7cdc767987664cada71b4f493fca8533)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:23:48.069618 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efd06a8-debb-4ca4-a7ca-abac52ae6ca7 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:23:48.116343 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-6efd06a8-debb-4ca4-a7ca-abac52ae6ca7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7cdc767987664cada71b4f493fca8533', 'request_id': 'req-6efd06a8-debb-4ca4-a7ca-abac52ae6ca7'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:23:48.122697 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-6efd06a8-debb-4ca4-a7ca-abac52ae6ca7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7cdc767987664cada71b4f493fca8533', 'request_id': 'req-6efd06a8-debb-4ca4-a7ca-abac52ae6ca7'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:23:48.128844 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-6efd06a8-debb-4ca4-a7ca-abac52ae6ca7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7cdc767987664cada71b4f493fca8533', 'request_id': 'req-6efd06a8-debb-4ca4-a7ca-abac52ae6ca7'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:23:48.137864 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:23:48.142947 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-6efd06a8-debb-4ca4-a7ca-abac52ae6ca7 admin admin] The token cache is being invalidate because project 7cdc767987664cada71b4f493fca8533 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80381) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 09:23:48.143262 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-6efd06a8-debb-4ca4-a7ca-abac52ae6ca7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:23:48.143965 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-6efd06a8-debb-4ca4-a7ca-abac52ae6ca7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:23:48.146268 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1198/2417] 10.222.0.26 () {58 vars in 1232 bytes} [Thu Jul 14 09:23:47 2022] DELETE /identity/v3/projects/7cdc767987664cada71b4f493fca8533 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 09:23:48.151405 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03148a39-aaee-4dd7-b093-9c74e83103a2 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:23:48.151690 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03148a39-aaee-4dd7-b093-9c74e83103a2 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:23:48.151965 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03148a39-aaee-4dd7-b093-9c74e83103a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:23:48.170056 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-03148a39-aaee-4dd7-b093-9c74e83103a2 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:23:48.171151 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-03148a39-aaee-4dd7-b093-9c74e83103a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:23:48.230903 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1220/2418] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:23:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:23:48.238726 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1045994-74c7-40de-abbf-4536dacd09f1 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:23:48.239014 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1045994-74c7-40de-abbf-4536dacd09f1 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:23:48.239323 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1045994-74c7-40de-abbf-4536dacd09f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:23:48.260343 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-c1045994-74c7-40de-abbf-4536dacd09f1 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:23:48.261546 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-c1045994-74c7-40de-abbf-4536dacd09f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:23:48.269556 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1199/2419] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:23:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:23:48.274022 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03148a39-aaee-4dd7-b093-9c74e83103a2 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:23:48.284544 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03148a39-aaee-4dd7-b093-9c74e83103a2 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:23:48.289898 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79901bc7-cd72-4fd5-af4b-56dbcb2ae846 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:23:48.290584 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79901bc7-cd72-4fd5-af4b-56dbcb2ae846 admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:23:48.290862 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79901bc7-cd72-4fd5-af4b-56dbcb2ae846 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:23:48.291133 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79901bc7-cd72-4fd5-af4b-56dbcb2ae846 admin admin] PATH_INFO: `/v3/domains` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:23:48.291896 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79901bc7-cd72-4fd5-af4b-56dbcb2ae846 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:23:48.292202 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79901bc7-cd72-4fd5-af4b-56dbcb2ae846 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:23:48.293814 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79901bc7-cd72-4fd5-af4b-56dbcb2ae846 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:23:48.306412 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1221/2420] 10.222.0.26 () {58 vars in 1152 bytes} [Thu Jul 14 09:23:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:23:48.310597 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1045994-74c7-40de-abbf-4536dacd09f1 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:23:48.319964 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1045994-74c7-40de-abbf-4536dacd09f1 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:23:48.324647 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedf20f1-5d42-4360-9a2c-34d7a2c20762 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:23:48.325330 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedf20f1-5d42-4360-9a2c-34d7a2c20762 admin admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:23:48.325609 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedf20f1-5d42-4360-9a2c-34d7a2c20762 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:23:48.325877 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedf20f1-5d42-4360-9a2c-34d7a2c20762 admin admin] PATH_INFO: `/v3/projects` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:23:48.326567 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedf20f1-5d42-4360-9a2c-34d7a2c20762 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:23:48.328521 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedf20f1-5d42-4360-9a2c-34d7a2c20762 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:23:48.359892 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1200/2421] 10.222.0.26 () {60 vars in 1152 bytes} [Thu Jul 14 09:23:48 2022] POST /identity/v3/projects => generated 383 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 09:23:48.361343 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79901bc7-cd72-4fd5-af4b-56dbcb2ae846 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:23:48.372164 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79901bc7-cd72-4fd5-af4b-56dbcb2ae846 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:23:48.418753 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:23:48.440473 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dc92df-993c-4f28-bb49-2605a3416a8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:23:48.441194 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dc92df-993c-4f28-bb49-2605a3416a8b admin admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:23:48.441475 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dc92df-993c-4f28-bb49-2605a3416a8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:23:48.441749 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dc92df-993c-4f28-bb49-2605a3416a8b admin admin] PATH_INFO: `/v3/users` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:23:48.442442 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49dc92df-993c-4f28-bb49-2605a3416a8b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:23:48.444553 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49dc92df-993c-4f28-bb49-2605a3416a8b admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:23:48.471989 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1222/2422] 10.222.0.26 () {60 vars in 1143 bytes} [Thu Jul 14 09:23:48 2022] POST /identity/v3/users => generated 292 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 09:23:48.475987 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedf20f1-5d42-4360-9a2c-34d7a2c20762 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:23:48.492605 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedf20f1-5d42-4360-9a2c-34d7a2c20762 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:23:48.498403 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889fe986-972c-4532-bf8b-fa190c161e6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:23:48.498985 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889fe986-972c-4532-bf8b-fa190c161e6e admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:23:48.499321 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889fe986-972c-4532-bf8b-fa190c161e6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:23:48.499555 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889fe986-972c-4532-bf8b-fa190c161e6e admin admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:23:48.500287 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889fe986-972c-4532-bf8b-fa190c161e6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:23:48.501877 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889fe986-972c-4532-bf8b-fa190c161e6e admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:23:48.513378 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1201/2423] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jul 14 09:23:48 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 09:23:48.517733 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dc92df-993c-4f28-bb49-2605a3416a8b admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:23:48.528529 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dc92df-993c-4f28-bb49-2605a3416a8b admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:23:48.533232 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed929089-3921-4a0c-a694-8849f931ebf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:23:48.533903 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed929089-3921-4a0c-a694-8849f931ebf6 admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:23:48.534182 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed929089-3921-4a0c-a694-8849f931ebf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:23:48.534456 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed929089-3921-4a0c-a694-8849f931ebf6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:23:48.535202 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed929089-3921-4a0c-a694-8849f931ebf6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:23:48.536817 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed929089-3921-4a0c-a694-8849f931ebf6 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:23:48.548379 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1223/2424] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jul 14 09:23:48 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 09:23:48.555034 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889fe986-972c-4532-bf8b-fa190c161e6e admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:23:48.565252 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889fe986-972c-4532-bf8b-fa190c161e6e admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:23:48.570038 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282d9e0a-597b-4633-bc54-1fc24a98e147 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:23:48.570924 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282d9e0a-597b-4633-bc54-1fc24a98e147 admin admin] REQUEST_METHOD: `PUT` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:23:48.571234 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282d9e0a-597b-4633-bc54-1fc24a98e147 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:23:48.571659 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282d9e0a-597b-4633-bc54-1fc24a98e147 admin admin] PATH_INFO: `/v3/projects/335a0619bd62458783523da604a22d40/users/42662fb9bae5403a8a6f698358f5c20c/roles/d1691322ce6a4edab7ce722edf0f2988` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:23:48.588584 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282d9e0a-597b-4633-bc54-1fc24a98e147 admin admin] RBAC: Authorizing `identity:create_grant(project_id=335a0619bd62458783523da604a22d40, user_id=42662fb9bae5403a8a6f698358f5c20c, role_id=d1691322ce6a4edab7ce722edf0f2988)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:23:48.591647 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282d9e0a-597b-4633-bc54-1fc24a98e147 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:23:48.610620 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1202/2425] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jul 14 09:23:48 2022] PUT /identity/v3/projects/335a0619bd62458783523da604a22d40/users/42662fb9bae5403a8a6f698358f5c20c/roles/d1691322ce6a4edab7ce722edf0f2988 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:23:48.617040 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45651dd9-073d-4795-8e46-fbdef35d8520 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:23:48.617331 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45651dd9-073d-4795-8e46-fbdef35d8520 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:23:48.617611 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45651dd9-073d-4795-8e46-fbdef35d8520 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:23:48.640779 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-45651dd9-073d-4795-8e46-fbdef35d8520 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:23:48.641979 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-45651dd9-073d-4795-8e46-fbdef35d8520 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:23:48.705269 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1224/2426] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:23:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:23:48.715106 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282d9e0a-597b-4633-bc54-1fc24a98e147 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:23:48.716990 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-282d9e0a-597b-4633-bc54-1fc24a98e147 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:23:48.775749 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282d9e0a-597b-4633-bc54-1fc24a98e147 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:23:48.779446 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2651653d-23b4-40f1-bffd-4becbde7c96f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '952911c492444ffea4b211a24b564117', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:23:48.780102 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2651653d-23b4-40f1-bffd-4becbde7c96f service neutron] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:23:48.780388 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2651653d-23b4-40f1-bffd-4becbde7c96f service neutron] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:23:48.780662 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2651653d-23b4-40f1-bffd-4becbde7c96f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:23:48.790168 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2651653d-23b4-40f1-bffd-4becbde7c96f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:23:48.791693 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2651653d-23b4-40f1-bffd-4becbde7c96f service neutron] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:23:48.801721 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1203/2427] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:23:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:23:49.416413 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d82b15-e2dc-48a1-ba08-8715ff7c8bb3 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:23:49.416707 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d82b15-e2dc-48a1-ba08-8715ff7c8bb3 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:23:49.417011 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d82b15-e2dc-48a1-ba08-8715ff7c8bb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:23:49.446375 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-89d82b15-e2dc-48a1-ba08-8715ff7c8bb3 None None] MFA Rules not processed for user `42662fb9bae5403a8a6f698358f5c20c`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:23:49.447721 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-89d82b15-e2dc-48a1-ba08-8715ff7c8bb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:23:49.517459 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1225/2428] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:23:49 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:23:49.529738 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2651653d-23b4-40f1-bffd-4becbde7c96f service neutron] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:23:49.531249 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-2651653d-23b4-40f1-bffd-4becbde7c96f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:23:49.601516 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2651653d-23b4-40f1-bffd-4becbde7c96f service neutron] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:23:49.605083 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fa23db-d6e4-40f9-a1a3-cf9b6efe4875 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e76f1952a5b146808844b9f2b7395035', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:23:49.605727 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fa23db-d6e4-40f9-a1a3-cf9b6efe4875 service cinder] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:23:49.606000 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fa23db-d6e4-40f9-a1a3-cf9b6efe4875 service cinder] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:23:49.606279 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fa23db-d6e4-40f9-a1a3-cf9b6efe4875 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:23:49.614471 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fa23db-d6e4-40f9-a1a3-cf9b6efe4875 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:23:49.615924 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fa23db-d6e4-40f9-a1a3-cf9b6efe4875 service cinder] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:23:49.624898 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1204/2429] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:23:49 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:23:49.820569 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d82b15-e2dc-48a1-ba08-8715ff7c8bb3 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:23:49.822598 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-89d82b15-e2dc-48a1-ba08-8715ff7c8bb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:23:49.892451 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:23:49.919340 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d82b15-e2dc-48a1-ba08-8715ff7c8bb3 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:23:49.930053 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cd8b52-c73f-4dfd-b43a-f81a3be847bf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '538fff3c5a2b487eb44ae7c1a1198102', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:23:49.930053 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cd8b52-c73f-4dfd-b43a-f81a3be847bf service nova] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:23:49.930053 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cd8b52-c73f-4dfd-b43a-f81a3be847bf service nova] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:23:49.930053 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cd8b52-c73f-4dfd-b43a-f81a3be847bf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:23:49.930727 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-a3cd8b52-c73f-4dfd-b43a-f81a3be847bf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:23:49.944755 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cd8b52-c73f-4dfd-b43a-f81a3be847bf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:23:49.946389 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cd8b52-c73f-4dfd-b43a-f81a3be847bf service nova] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:23:49.959224 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1226/2430] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:23:49 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:23:50.970970 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d53164e-5fbd-408c-bdcd-affe818c6b35 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:23:50.971508 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d53164e-5fbd-408c-bdcd-affe818c6b35 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:23:50.971614 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d53164e-5fbd-408c-bdcd-affe818c6b35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:23:50.990636 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b9d548-5e62-447f-b936-c34d2d038094 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:23:50.991213 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b9d548-5e62-447f-b936-c34d2d038094 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:23:50.991317 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b9d548-5e62-447f-b936-c34d2d038094 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:23:50.996260 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-5d53164e-5fbd-408c-bdcd-affe818c6b35 None None] MFA Rules not processed for user `7d6267ede8d64e2abe1f4058a9797fb7`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:23:50.997525 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-5d53164e-5fbd-408c-bdcd-affe818c6b35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:23:51.012117 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-17b9d548-5e62-447f-b936-c34d2d038094 None None] MFA Rules not processed for user `42662fb9bae5403a8a6f698358f5c20c`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:23:51.013412 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-17b9d548-5e62-447f-b936-c34d2d038094 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:23:51.021014 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1227/2431] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:23:50 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 09:23:51.035548 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:23:51.069768 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1205/2432] 10.222.0.26 () {58 vars in 1009 bytes} [Thu Jul 14 09:23:50 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:23:51.082866 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b9d548-5e62-447f-b936-c34d2d038094 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:23:51.084899 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-17b9d548-5e62-447f-b936-c34d2d038094 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:23:51.160833 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b9d548-5e62-447f-b936-c34d2d038094 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:23:51.165979 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b479794e-cb90-4a25-86ef-d7f2d5d7f845 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5a43c80514b4a13af03c7f8ebcf7a5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:23:51.166746 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b479794e-cb90-4a25-86ef-d7f2d5d7f845 service swift] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:23:51.167056 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b479794e-cb90-4a25-86ef-d7f2d5d7f845 service swift] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:23:51.167390 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b479794e-cb90-4a25-86ef-d7f2d5d7f845 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:23:51.176493 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b479794e-cb90-4a25-86ef-d7f2d5d7f845 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:23:51.178124 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b479794e-cb90-4a25-86ef-d7f2d5d7f845 service swift] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:23:51.187375 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1228/2433] 10.222.0.26 () {60 vars in 1384 bytes} [Thu Jul 14 09:23:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 14 09:24:01.059644 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=7 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:24:01.184612 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=44 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:24:05.154181 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d53164e-5fbd-408c-bdcd-affe818c6b35 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:24:05.172327 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d53164e-5fbd-408c-bdcd-affe818c6b35 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:24:05.175930 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba7c831-935b-46b6-a812-5cde7e31cbb5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '952911c492444ffea4b211a24b564117', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:24:05.176602 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba7c831-935b-46b6-a812-5cde7e31cbb5 service neutron] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:24:05.176872 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba7c831-935b-46b6-a812-5cde7e31cbb5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:24:05.177152 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba7c831-935b-46b6-a812-5cde7e31cbb5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:24:05.178937 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-cba7c831-935b-46b6-a812-5cde7e31cbb5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:24:05.189599 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba7c831-935b-46b6-a812-5cde7e31cbb5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:24:05.191036 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba7c831-935b-46b6-a812-5cde7e31cbb5 service neutron] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:24:05.199340 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1206/2434] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:24:05 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:24:05.309277 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6983a3e2-d9e6-4ac6-bd47-114205b6fccb None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:24:05.310575 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6983a3e2-d9e6-4ac6-bd47-114205b6fccb None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:24:05.310941 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6983a3e2-d9e6-4ac6-bd47-114205b6fccb None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:24:05.312627 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1229/2435] 10.222.0.26 () {54 vars in 921 bytes} [Thu Jul 14 09:24:05 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:24:05.322168 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad6502a-7320-4ec7-ab83-f07ac7c54bfc None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:24:05.322457 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad6502a-7320-4ec7-ab83-f07ac7c54bfc None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:24:05.322743 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad6502a-7320-4ec7-ab83-f07ac7c54bfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:24:05.354496 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-1ad6502a-7320-4ec7-ab83-f07ac7c54bfc None None] MFA Rules not processed for user `538fff3c5a2b487eb44ae7c1a1198102`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:24:05.355913 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-1ad6502a-7320-4ec7-ab83-f07ac7c54bfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:24:05.368739 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1207/2436] 10.222.0.26 () {58 vars in 1018 bytes} [Thu Jul 14 09:24:05 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:24:05.382863 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6983a3e2-d9e6-4ac6-bd47-114205b6fccb None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:24:05.384839 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-6983a3e2-d9e6-4ac6-bd47-114205b6fccb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:24:05.479720 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6983a3e2-d9e6-4ac6-bd47-114205b6fccb None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:24:05.483766 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6667a971-1087-4e65-813a-c4e3581d7ab6 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '872dbb91c9e54f13b1a31fc8242126df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:24:05.484554 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6667a971-1087-4e65-813a-c4e3581d7ab6 service glance] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:24:05.484933 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6667a971-1087-4e65-813a-c4e3581d7ab6 service glance] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:24:05.485301 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6667a971-1087-4e65-813a-c4e3581d7ab6 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:24:05.499973 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6667a971-1087-4e65-813a-c4e3581d7ab6 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:24:05.501502 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6667a971-1087-4e65-813a-c4e3581d7ab6 service glance] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:24:05.514028 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1230/2437] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:24:05 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:24:05.582903 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093ee8a2-a080-4d9e-9ef3-d2823561ff32 None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:24:05.583767 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093ee8a2-a080-4d9e-9ef3-d2823561ff32 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:24:05.584189 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093ee8a2-a080-4d9e-9ef3-d2823561ff32 None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:24:05.586450 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1208/2438] 10.222.0.26 () {54 vars in 921 bytes} [Thu Jul 14 09:24:05 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:24:05.592014 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553b8aeb-735f-45f4-abb0-6a610fd55679 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:24:05.592316 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553b8aeb-735f-45f4-abb0-6a610fd55679 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:24:05.592685 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553b8aeb-735f-45f4-abb0-6a610fd55679 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:24:05.611555 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-553b8aeb-735f-45f4-abb0-6a610fd55679 None None] MFA Rules not processed for user `952911c492444ffea4b211a24b564117`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:24:05.612820 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-553b8aeb-735f-45f4-abb0-6a610fd55679 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:24:05.619544 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1231/2439] 10.222.0.26 () {58 vars in 1018 bytes} [Thu Jul 14 09:24:05 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:24:05.632378 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093ee8a2-a080-4d9e-9ef3-d2823561ff32 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:24:05.644512 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093ee8a2-a080-4d9e-9ef3-d2823561ff32 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:24:05.648482 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bd3ddb-be06-4247-8869-67538b4ba657 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '952911c492444ffea4b211a24b564117', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:24:05.649392 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bd3ddb-be06-4247-8869-67538b4ba657 service neutron] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:24:05.649744 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bd3ddb-be06-4247-8869-67538b4ba657 service neutron] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:24:05.650088 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bd3ddb-be06-4247-8869-67538b4ba657 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:24:05.658692 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bd3ddb-be06-4247-8869-67538b4ba657 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:24:05.660326 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bd3ddb-be06-4247-8869-67538b4ba657 service neutron] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:24:05.669796 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1209/2440] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:24:05 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:24:05.758009 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06609325-ba0f-44e7-b824-04c754529436 None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:24:05.758318 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06609325-ba0f-44e7-b824-04c754529436 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:24:05.759411 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06609325-ba0f-44e7-b824-04c754529436 None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:24:05.761291 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1232/2441] 10.222.0.26 () {54 vars in 921 bytes} [Thu Jul 14 09:24:05 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:24:05.767352 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563ff6eb-91d3-4475-9bc3-82d606d7fc94 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:24:05.767727 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563ff6eb-91d3-4475-9bc3-82d606d7fc94 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:24:05.768097 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563ff6eb-91d3-4475-9bc3-82d606d7fc94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:24:05.788589 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-563ff6eb-91d3-4475-9bc3-82d606d7fc94 None None] MFA Rules not processed for user `538fff3c5a2b487eb44ae7c1a1198102`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:24:05.789982 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-563ff6eb-91d3-4475-9bc3-82d606d7fc94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:24:05.800360 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1210/2442] 10.222.0.26 () {58 vars in 1018 bytes} [Thu Jul 14 09:24:05 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 09:24:05.810919 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06609325-ba0f-44e7-b824-04c754529436 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:24:05.823026 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06609325-ba0f-44e7-b824-04c754529436 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:24:05.826746 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29da5b12-8e5f-4672-846f-bd5aed60d125 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '952911c492444ffea4b211a24b564117', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:24:05.827448 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29da5b12-8e5f-4672-846f-bd5aed60d125 service neutron] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:24:05.827726 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29da5b12-8e5f-4672-846f-bd5aed60d125 service neutron] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:24:05.828006 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29da5b12-8e5f-4672-846f-bd5aed60d125 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:24:05.839763 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29da5b12-8e5f-4672-846f-bd5aed60d125 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:24:05.841336 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29da5b12-8e5f-4672-846f-bd5aed60d125 service neutron] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:24:05.855845 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1233/2443] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:24:05 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:24:08.677398 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b585056-8df1-437f-a3be-b9b7d31bce89 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:24:08.677869 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b585056-8df1-437f-a3be-b9b7d31bce89 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:24:08.677974 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b585056-8df1-437f-a3be-b9b7d31bce89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:24:08.699641 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-9b585056-8df1-437f-a3be-b9b7d31bce89 None None] MFA Rules not processed for user `7d6267ede8d64e2abe1f4058a9797fb7`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:24:08.701027 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-9b585056-8df1-437f-a3be-b9b7d31bce89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:24:08.708629 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1211/2444] 10.222.0.26 () {58 vars in 1009 bytes} [Thu Jul 14 09:24:08 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:24:08.720671 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29da5b12-8e5f-4672-846f-bd5aed60d125 service neutron] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:24:08.733366 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29da5b12-8e5f-4672-846f-bd5aed60d125 service neutron] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:24:08.739117 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1cba7d-79bb-4415-8c4f-6795f15eede1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5a43c80514b4a13af03c7f8ebcf7a5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:24:08.739857 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1cba7d-79bb-4415-8c4f-6795f15eede1 service swift] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:24:08.740141 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1cba7d-79bb-4415-8c4f-6795f15eede1 service swift] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:24:08.740418 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1cba7d-79bb-4415-8c4f-6795f15eede1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:24:08.749209 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1cba7d-79bb-4415-8c4f-6795f15eede1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:24:08.750823 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1cba7d-79bb-4415-8c4f-6795f15eede1 service swift] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:24:08.766744 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1234/2445] 10.222.0.26 () {60 vars in 1384 bytes} [Thu Jul 14 09:24:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 14 09:24:10.201458 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b585056-8df1-437f-a3be-b9b7d31bce89 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:24:10.202982 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-9b585056-8df1-437f-a3be-b9b7d31bce89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:24:10.219767 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b585056-8df1-437f-a3be-b9b7d31bce89 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:24:10.226195 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76eaff6-dc1d-4b83-854d-ca4a6b7f9e46 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '538fff3c5a2b487eb44ae7c1a1198102', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:24:10.226850 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76eaff6-dc1d-4b83-854d-ca4a6b7f9e46 service nova] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:24:10.227209 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76eaff6-dc1d-4b83-854d-ca4a6b7f9e46 service nova] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:24:10.227872 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76eaff6-dc1d-4b83-854d-ca4a6b7f9e46 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:24:10.230595 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-b76eaff6-dc1d-4b83-854d-ca4a6b7f9e46 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:24:10.310501 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76eaff6-dc1d-4b83-854d-ca4a6b7f9e46 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:24:10.312297 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76eaff6-dc1d-4b83-854d-ca4a6b7f9e46 service nova] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:24:10.323716 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1212/2446] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:24:10 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:24:18.758545 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=41 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:24:20.318265 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=49 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:24:26.530357 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947ec885-d429-455d-bb4a-665d442ee5bc None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:24:26.530820 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947ec885-d429-455d-bb4a-665d442ee5bc None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:24:26.530900 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947ec885-d429-455d-bb4a-665d442ee5bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:24:26.551002 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-947ec885-d429-455d-bb4a-665d442ee5bc None None] MFA Rules not processed for user `7d6267ede8d64e2abe1f4058a9797fb7`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:24:26.552306 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-947ec885-d429-455d-bb4a-665d442ee5bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:24:26.560661 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1235/2447] 10.222.0.26 () {58 vars in 967 bytes} [Thu Jul 14 09:24:26 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:24:26.588461 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76eaff6-dc1d-4b83-854d-ca4a6b7f9e46 service nova] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:24:26.600760 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76eaff6-dc1d-4b83-854d-ca4a6b7f9e46 service nova] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:24:26.606675 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9937bc-d6e8-4981-9739-d0717dd608eb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5a43c80514b4a13af03c7f8ebcf7a5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:24:26.607391 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9937bc-d6e8-4981-9739-d0717dd608eb service swift] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:24:26.607682 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9937bc-d6e8-4981-9739-d0717dd608eb service swift] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:24:26.607968 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9937bc-d6e8-4981-9739-d0717dd608eb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:24:26.616379 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9937bc-d6e8-4981-9739-d0717dd608eb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:24:26.617995 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9937bc-d6e8-4981-9739-d0717dd608eb service swift] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:24:26.626144 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1213/2448] 10.222.0.26 () {60 vars in 1384 bytes} [Thu Jul 14 09:24:26 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 14 09:24:29.458494 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947ec885-d429-455d-bb4a-665d442ee5bc None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:24:29.468030 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947ec885-d429-455d-bb4a-665d442ee5bc None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:24:29.472664 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da955b28-9865-411b-ad26-c0447c0c0940 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '952911c492444ffea4b211a24b564117', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:24:29.473305 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da955b28-9865-411b-ad26-c0447c0c0940 service neutron] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:24:29.473626 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da955b28-9865-411b-ad26-c0447c0c0940 service neutron] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:24:29.473887 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da955b28-9865-411b-ad26-c0447c0c0940 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:24:29.476099 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-da955b28-9865-411b-ad26-c0447c0c0940 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:24:29.489754 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da955b28-9865-411b-ad26-c0447c0c0940 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:24:29.491208 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da955b28-9865-411b-ad26-c0447c0c0940 service neutron] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:24:29.502040 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1236/2449] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:24:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:24:29.687628 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9937bc-d6e8-4981-9739-d0717dd608eb service swift] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:24:29.689099 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-6c9937bc-d6e8-4981-9739-d0717dd608eb service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:24:29.701187 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9937bc-d6e8-4981-9739-d0717dd608eb service swift] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:24:29.705030 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78439708-1b03-484f-a8c2-4b02e24c27f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '952911c492444ffea4b211a24b564117', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:24:29.705675 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78439708-1b03-484f-a8c2-4b02e24c27f7 service neutron] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:24:29.705962 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78439708-1b03-484f-a8c2-4b02e24c27f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:24:29.706239 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78439708-1b03-484f-a8c2-4b02e24c27f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:24:29.708167 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-78439708-1b03-484f-a8c2-4b02e24c27f7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:24:29.720400 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78439708-1b03-484f-a8c2-4b02e24c27f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:24:29.721813 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78439708-1b03-484f-a8c2-4b02e24c27f7 service neutron] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:24:29.731933 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1214/2450] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:24:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 09:24:32.855855 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da955b28-9865-411b-ad26-c0447c0c0940 service neutron] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:24:32.858240 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-da955b28-9865-411b-ad26-c0447c0c0940 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:24:32.874898 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da955b28-9865-411b-ad26-c0447c0c0940 service neutron] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:24:32.881129 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba5fda4-e495-4a15-8bf1-c7489d455321 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:24:32.882023 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba5fda4-e495-4a15-8bf1-c7489d455321 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:24:32.882304 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba5fda4-e495-4a15-8bf1-c7489d455321 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:24:32.882577 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba5fda4-e495-4a15-8bf1-c7489d455321 admin admin] PATH_INFO: `/v3/users/a2ab766e96024b899a01c540d9bd1322` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:24:32.883803 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba5fda4-e495-4a15-8bf1-c7489d455321 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a2ab766e96024b899a01c540d9bd1322)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:24:32.885857 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba5fda4-e495-4a15-8bf1-c7489d455321 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:24:32.951819 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-9ba5fda4-e495-4a15-8bf1-c7489d455321 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a2ab766e96024b899a01c540d9bd1322'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:24:32.952570 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-9ba5fda4-e495-4a15-8bf1-c7489d455321 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a2ab766e96024b899a01c540d9bd1322'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:24:32.969511 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-9ba5fda4-e495-4a15-8bf1-c7489d455321 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a2ab766e96024b899a01c540d9bd1322'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:24:32.980143 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-9ba5fda4-e495-4a15-8bf1-c7489d455321 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a2ab766e96024b899a01c540d9bd1322'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:24:32.980855 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-9ba5fda4-e495-4a15-8bf1-c7489d455321 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a2ab766e96024b899a01c540d9bd1322'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:24:32.995159 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1237/2451] 10.222.0.26 () {58 vars in 1223 bytes} [Thu Jul 14 09:24:32 2022] DELETE /identity/v3/users/a2ab766e96024b899a01c540d9bd1322 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:24:33.305594 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78439708-1b03-484f-a8c2-4b02e24c27f7 service neutron] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:24:33.307048 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-78439708-1b03-484f-a8c2-4b02e24c27f7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:24:33.389309 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba5fda4-e495-4a15-8bf1-c7489d455321 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:24:33.391892 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-9ba5fda4-e495-4a15-8bf1-c7489d455321 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:24:33.461212 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78439708-1b03-484f-a8c2-4b02e24c27f7 service neutron] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:24:33.492699 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba5fda4-e495-4a15-8bf1-c7489d455321 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:24:33.504552 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605bfd5f-ee39-41af-93c0-93f05b855444 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:24:33.505834 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605bfd5f-ee39-41af-93c0-93f05b855444 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:24:33.505834 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605bfd5f-ee39-41af-93c0-93f05b855444 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:24:33.506270 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605bfd5f-ee39-41af-93c0-93f05b855444 admin admin] PATH_INFO: `/v3/users/4d619bfc0939493480fa98451db2f30f` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:24:33.508231 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605bfd5f-ee39-41af-93c0-93f05b855444 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4d619bfc0939493480fa98451db2f30f)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:24:33.510314 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605bfd5f-ee39-41af-93c0-93f05b855444 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:24:33.550331 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce4cd91-9ddc-4452-8f73-8a99976cabf8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '355684f52e074ad9bfa7fd90bcae6caf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:24:33.551715 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce4cd91-9ddc-4452-8f73-8a99976cabf8 service placement] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:24:33.552156 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce4cd91-9ddc-4452-8f73-8a99976cabf8 service placement] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:24:33.552995 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce4cd91-9ddc-4452-8f73-8a99976cabf8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:24:33.555913 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-8ce4cd91-9ddc-4452-8f73-8a99976cabf8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:24:33.570910 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce4cd91-9ddc-4452-8f73-8a99976cabf8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:24:33.572967 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce4cd91-9ddc-4452-8f73-8a99976cabf8 service placement] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:24:33.584695 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1215/2452] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:24:33 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 280 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:24:33.585809 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-605bfd5f-ee39-41af-93c0-93f05b855444 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4d619bfc0939493480fa98451db2f30f'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:24:33.585961 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce4cd91-9ddc-4452-8f73-8a99976cabf8 service placement] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:24:33.587432 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-605bfd5f-ee39-41af-93c0-93f05b855444 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4d619bfc0939493480fa98451db2f30f'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:24:33.587602 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-8ce4cd91-9ddc-4452-8f73-8a99976cabf8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:24:33.593177 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=81,keystone:DELETE=18,keystone:INSERT=1 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:24:33.602488 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-605bfd5f-ee39-41af-93c0-93f05b855444 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4d619bfc0939493480fa98451db2f30f'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:24:33.606296 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce4cd91-9ddc-4452-8f73-8a99976cabf8 service placement] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:24:33.613843 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ce992e-b4cc-4232-b226-2ba0463689e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:24:33.614204 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-605bfd5f-ee39-41af-93c0-93f05b855444 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4d619bfc0939493480fa98451db2f30f'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:24:33.614736 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ce992e-b4cc-4232-b226-2ba0463689e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:24:33.615179 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ce992e-b4cc-4232-b226-2ba0463689e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:24:33.615508 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-605bfd5f-ee39-41af-93c0-93f05b855444 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4d619bfc0939493480fa98451db2f30f'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:24:33.615634 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ce992e-b4cc-4232-b226-2ba0463689e7 admin admin] PATH_INFO: `/v3/projects/9b8c0df760e04c5ca5f000c048f1da4c` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:24:33.616710 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ce992e-b4cc-4232-b226-2ba0463689e7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9b8c0df760e04c5ca5f000c048f1da4c)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:24:33.618752 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ce992e-b4cc-4232-b226-2ba0463689e7 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:24:33.632244 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1238/2453] 10.222.0.26 () {58 vars in 1223 bytes} [Thu Jul 14 09:24:33 2022] DELETE /identity/v3/users/4d619bfc0939493480fa98451db2f30f => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:24:33.668652 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-85ce992e-b4cc-4232-b226-2ba0463689e7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9b8c0df760e04c5ca5f000c048f1da4c', 'request_id': 'req-85ce992e-b4cc-4232-b226-2ba0463689e7'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:24:33.674955 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-85ce992e-b4cc-4232-b226-2ba0463689e7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9b8c0df760e04c5ca5f000c048f1da4c', 'request_id': 'req-85ce992e-b4cc-4232-b226-2ba0463689e7'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:24:33.680872 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:24:33.683698 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-85ce992e-b4cc-4232-b226-2ba0463689e7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9b8c0df760e04c5ca5f000c048f1da4c', 'request_id': 'req-85ce992e-b4cc-4232-b226-2ba0463689e7'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:24:33.698049 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-85ce992e-b4cc-4232-b226-2ba0463689e7 admin admin] The token cache is being invalidate because project 9b8c0df760e04c5ca5f000c048f1da4c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80381) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 09:24:33.698268 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-85ce992e-b4cc-4232-b226-2ba0463689e7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:24:33.699004 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-85ce992e-b4cc-4232-b226-2ba0463689e7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:24:33.700621 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1216/2454] 10.222.0.26 () {58 vars in 1232 bytes} [Thu Jul 14 09:24:33 2022] DELETE /identity/v3/projects/9b8c0df760e04c5ca5f000c048f1da4c => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:24:33.705743 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee600cb9-4b3e-4efd-9942-80f4b20774e7 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:24:33.706040 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee600cb9-4b3e-4efd-9942-80f4b20774e7 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:24:33.706321 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee600cb9-4b3e-4efd-9942-80f4b20774e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:24:33.725172 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-ee600cb9-4b3e-4efd-9942-80f4b20774e7 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:24:33.726485 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-ee600cb9-4b3e-4efd-9942-80f4b20774e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:24:33.789407 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1239/2455] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:24:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:24:33.798062 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc86d10-64e3-49df-b34d-b7d375c92371 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:24:33.798348 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc86d10-64e3-49df-b34d-b7d375c92371 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:24:33.798672 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc86d10-64e3-49df-b34d-b7d375c92371 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:24:33.812122 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee600cb9-4b3e-4efd-9942-80f4b20774e7 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:24:33.813645 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-ee600cb9-4b3e-4efd-9942-80f4b20774e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:24:33.820247 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-4bc86d10-64e3-49df-b34d-b7d375c92371 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:24:33.821843 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-4bc86d10-64e3-49df-b34d-b7d375c92371 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:24:33.826880 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee600cb9-4b3e-4efd-9942-80f4b20774e7 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:24:33.831650 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef739f4-ddd2-42e8-96b4-e88195802195 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:24:33.834982 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1217/2456] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:24:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:24:33.835151 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef739f4-ddd2-42e8-96b4-e88195802195 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:24:33.835151 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef739f4-ddd2-42e8-96b4-e88195802195 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:24:33.835151 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef739f4-ddd2-42e8-96b4-e88195802195 admin admin] PATH_INFO: `/v3/projects/728ef81aa26445e6a42abdf976d146db` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:24:33.835151 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef739f4-ddd2-42e8-96b4-e88195802195 admin admin] RBAC: Authorizing `identity:delete_project(project_id=728ef81aa26445e6a42abdf976d146db)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:24:33.836097 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc86d10-64e3-49df-b34d-b7d375c92371 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:24:33.836845 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef739f4-ddd2-42e8-96b4-e88195802195 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:24:33.846393 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc86d10-64e3-49df-b34d-b7d375c92371 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:24:33.852019 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32724ac7-f678-4403-a54f-768df56a09ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:24:33.852717 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32724ac7-f678-4403-a54f-768df56a09ca admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:24:33.853006 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32724ac7-f678-4403-a54f-768df56a09ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:24:33.853281 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32724ac7-f678-4403-a54f-768df56a09ca admin admin] PATH_INFO: `/v3/domains` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:24:33.854011 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32724ac7-f678-4403-a54f-768df56a09ca admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:24:33.854324 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32724ac7-f678-4403-a54f-768df56a09ca admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:24:33.856022 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32724ac7-f678-4403-a54f-768df56a09ca admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:24:33.868754 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1218/2457] 10.222.0.26 () {58 vars in 1152 bytes} [Thu Jul 14 09:24:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:24:33.873208 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32724ac7-f678-4403-a54f-768df56a09ca admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:24:33.883724 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32724ac7-f678-4403-a54f-768df56a09ca admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:24:33.886615 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-3ef739f4-ddd2-42e8-96b4-e88195802195 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '728ef81aa26445e6a42abdf976d146db', 'request_id': 'req-3ef739f4-ddd2-42e8-96b4-e88195802195'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:24:33.893759 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-3ef739f4-ddd2-42e8-96b4-e88195802195 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '728ef81aa26445e6a42abdf976d146db', 'request_id': 'req-3ef739f4-ddd2-42e8-96b4-e88195802195'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:24:33.900717 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-3ef739f4-ddd2-42e8-96b4-e88195802195 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '728ef81aa26445e6a42abdf976d146db', 'request_id': 'req-3ef739f4-ddd2-42e8-96b4-e88195802195'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:24:33.914138 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-3ef739f4-ddd2-42e8-96b4-e88195802195 admin admin] The token cache is being invalidate because project 728ef81aa26445e6a42abdf976d146db was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80382) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 09:24:33.914427 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-3ef739f4-ddd2-42e8-96b4-e88195802195 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:24:33.915067 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-3ef739f4-ddd2-42e8-96b4-e88195802195 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:24:33.916808 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1240/2458] 10.222.0.26 () {58 vars in 1232 bytes} [Thu Jul 14 09:24:33 2022] DELETE /identity/v3/projects/728ef81aa26445e6a42abdf976d146db => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 09:24:33.921967 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7afe21d-91bd-4172-ad03-a4bc38aa5751 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:24:33.922256 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7afe21d-91bd-4172-ad03-a4bc38aa5751 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:24:33.922540 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7afe21d-91bd-4172-ad03-a4bc38aa5751 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:24:33.940944 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-b7afe21d-91bd-4172-ad03-a4bc38aa5751 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:24:33.942140 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-b7afe21d-91bd-4172-ad03-a4bc38aa5751 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:24:33.943040 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744f74ee-d09b-4f70-9fcf-c17cdff97452 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:24:33.943742 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744f74ee-d09b-4f70-9fcf-c17cdff97452 admin admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:24:33.944028 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744f74ee-d09b-4f70-9fcf-c17cdff97452 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:24:33.944306 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744f74ee-d09b-4f70-9fcf-c17cdff97452 admin admin] PATH_INFO: `/v3/projects` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:24:33.945035 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744f74ee-d09b-4f70-9fcf-c17cdff97452 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:24:33.947018 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744f74ee-d09b-4f70-9fcf-c17cdff97452 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:24:33.950417 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1241/2459] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:24:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 09:24:33.957932 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bd6754-124f-46c3-8c2e-59d9772c57f6 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:24:33.958224 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bd6754-124f-46c3-8c2e-59d9772c57f6 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:24:33.958506 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bd6754-124f-46c3-8c2e-59d9772c57f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:24:33.981393 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1219/2460] 10.222.0.26 () {60 vars in 1152 bytes} [Thu Jul 14 09:24:33 2022] POST /identity/v3/projects => generated 365 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 09:24:33.981634 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-58bd6754-124f-46c3-8c2e-59d9772c57f6 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:24:33.981634 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-58bd6754-124f-46c3-8c2e-59d9772c57f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:24:33.982694 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744f74ee-d09b-4f70-9fcf-c17cdff97452 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:24:33.984570 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-744f74ee-d09b-4f70-9fcf-c17cdff97452 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:24:34.026853 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:24:34.043523 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1242/2461] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:24:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 09:24:34.047662 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bd6754-124f-46c3-8c2e-59d9772c57f6 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:24:34.049992 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744f74ee-d09b-4f70-9fcf-c17cdff97452 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:24:34.054994 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d597960-a44e-492c-8039-97a196301db9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:24:34.055712 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d597960-a44e-492c-8039-97a196301db9 admin admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:24:34.056001 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d597960-a44e-492c-8039-97a196301db9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:24:34.056280 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d597960-a44e-492c-8039-97a196301db9 admin admin] PATH_INFO: `/v3/users` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:24:34.056524 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bd6754-124f-46c3-8c2e-59d9772c57f6 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:24:34.056975 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d597960-a44e-492c-8039-97a196301db9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:24:34.058981 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d597960-a44e-492c-8039-97a196301db9 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:24:34.061548 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4d5251-5fe4-464a-acc9-dc6f7bf71c10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:24:34.062201 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4d5251-5fe4-464a-acc9-dc6f7bf71c10 admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:24:34.062483 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4d5251-5fe4-464a-acc9-dc6f7bf71c10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:24:34.062757 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4d5251-5fe4-464a-acc9-dc6f7bf71c10 admin admin] PATH_INFO: `/v3/domains` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:24:34.063536 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4d5251-5fe4-464a-acc9-dc6f7bf71c10 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:24:34.063872 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4d5251-5fe4-464a-acc9-dc6f7bf71c10 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:24:34.065538 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4d5251-5fe4-464a-acc9-dc6f7bf71c10 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:24:34.078566 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1243/2462] 10.222.0.26 () {58 vars in 1152 bytes} [Thu Jul 14 09:24:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 09:24:34.082321 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1220/2463] 10.222.0.26 () {60 vars in 1143 bytes} [Thu Jul 14 09:24:33 2022] POST /identity/v3/users => generated 283 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 09:24:34.082549 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4d5251-5fe4-464a-acc9-dc6f7bf71c10 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:24:34.083123 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d597960-a44e-492c-8039-97a196301db9 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:24:34.093700 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4d5251-5fe4-464a-acc9-dc6f7bf71c10 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:24:34.094045 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d597960-a44e-492c-8039-97a196301db9 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:24:34.098758 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a16c2b5-fe66-45b7-bc42-89e5839bb14e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:24:34.098963 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6973056c-f48e-433c-8901-8221be8b7705 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:24:34.099448 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a16c2b5-fe66-45b7-bc42-89e5839bb14e admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:24:34.099755 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a16c2b5-fe66-45b7-bc42-89e5839bb14e admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:24:34.099881 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6973056c-f48e-433c-8901-8221be8b7705 admin admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:24:34.100027 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a16c2b5-fe66-45b7-bc42-89e5839bb14e admin admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:24:34.100131 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6973056c-f48e-433c-8901-8221be8b7705 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:24:34.100421 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6973056c-f48e-433c-8901-8221be8b7705 admin admin] PATH_INFO: `/v3/projects` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:24:34.100747 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a16c2b5-fe66-45b7-bc42-89e5839bb14e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:24:34.101113 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6973056c-f48e-433c-8901-8221be8b7705 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:24:34.102360 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a16c2b5-fe66-45b7-bc42-89e5839bb14e admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:24:34.103034 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6973056c-f48e-433c-8901-8221be8b7705 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:24:34.113367 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1221/2464] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jul 14 09:24:34 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 09:24:34.117284 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a16c2b5-fe66-45b7-bc42-89e5839bb14e admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:24:34.128942 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a16c2b5-fe66-45b7-bc42-89e5839bb14e admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:24:34.134024 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736a8615-c607-4704-b983-db23ffd5d057 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:24:34.134528 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1244/2465] 10.222.0.26 () {60 vars in 1152 bytes} [Thu Jul 14 09:24:34 2022] POST /identity/v3/projects => generated 383 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 09:24:34.134690 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736a8615-c607-4704-b983-db23ffd5d057 admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:24:34.134979 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736a8615-c607-4704-b983-db23ffd5d057 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:24:34.135294 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736a8615-c607-4704-b983-db23ffd5d057 admin admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:24:34.136024 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736a8615-c607-4704-b983-db23ffd5d057 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:24:34.137772 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736a8615-c607-4704-b983-db23ffd5d057 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:24:34.138463 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6973056c-f48e-433c-8901-8221be8b7705 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:24:34.148736 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1222/2466] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jul 14 09:24:34 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 09:24:34.152747 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736a8615-c607-4704-b983-db23ffd5d057 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:24:34.162493 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736a8615-c607-4704-b983-db23ffd5d057 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:24:34.181991 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:24:34.207417 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6973056c-f48e-433c-8901-8221be8b7705 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:24:34.212250 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0421b0e0-ced7-4e45-9de8-93feea2949d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:24:34.212925 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0421b0e0-ced7-4e45-9de8-93feea2949d4 admin admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:24:34.213205 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0421b0e0-ced7-4e45-9de8-93feea2949d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:24:34.213487 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0421b0e0-ced7-4e45-9de8-93feea2949d4 admin admin] PATH_INFO: `/v3/users` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:24:34.214205 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0421b0e0-ced7-4e45-9de8-93feea2949d4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:24:34.216159 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0421b0e0-ced7-4e45-9de8-93feea2949d4 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:24:34.222115 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e82c05c-5993-4251-91be-022cac2b6e8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:24:34.222989 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e82c05c-5993-4251-91be-022cac2b6e8e admin admin] REQUEST_METHOD: `PUT` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:24:34.223322 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e82c05c-5993-4251-91be-022cac2b6e8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:24:34.223602 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e82c05c-5993-4251-91be-022cac2b6e8e admin admin] PATH_INFO: `/v3/projects/6cb67ef188a244779657a2c465ab4fe2/users/f38291f7d1024c08b364e223ebf686f6/roles/d1691322ce6a4edab7ce722edf0f2988` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:24:34.239405 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1245/2467] 10.222.0.26 () {60 vars in 1143 bytes} [Thu Jul 14 09:24:34 2022] POST /identity/v3/users => generated 292 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 09:24:34.239757 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e82c05c-5993-4251-91be-022cac2b6e8e admin admin] RBAC: Authorizing `identity:create_grant(project_id=6cb67ef188a244779657a2c465ab4fe2, user_id=f38291f7d1024c08b364e223ebf686f6, role_id=d1691322ce6a4edab7ce722edf0f2988)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:24:34.242261 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e82c05c-5993-4251-91be-022cac2b6e8e admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:24:34.244144 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0421b0e0-ced7-4e45-9de8-93feea2949d4 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:24:34.256285 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0421b0e0-ced7-4e45-9de8-93feea2949d4 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:24:34.267882 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1223/2468] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jul 14 09:24:34 2022] PUT /identity/v3/projects/6cb67ef188a244779657a2c465ab4fe2/users/f38291f7d1024c08b364e223ebf686f6/roles/d1691322ce6a4edab7ce722edf0f2988 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 09:24:34.269520 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee6f6ef-1545-4aeb-b99d-529a497290a0 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:24:34.269973 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee6f6ef-1545-4aeb-b99d-529a497290a0 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:24:34.270569 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee6f6ef-1545-4aeb-b99d-529a497290a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:24:34.303711 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-fee6f6ef-1545-4aeb-b99d-529a497290a0 None None] MFA Rules not processed for user `f38291f7d1024c08b364e223ebf686f6`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:24:34.304071 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-fee6f6ef-1545-4aeb-b99d-529a497290a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:24:34.342857 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbebc42-ca66-4ae0-a0f7-d7f48ea1a5dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:24:34.343582 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbebc42-ca66-4ae0-a0f7-d7f48ea1a5dc admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:24:34.343864 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbebc42-ca66-4ae0-a0f7-d7f48ea1a5dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:24:34.344136 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbebc42-ca66-4ae0-a0f7-d7f48ea1a5dc admin admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:24:34.344869 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbebc42-ca66-4ae0-a0f7-d7f48ea1a5dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:24:34.346616 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbebc42-ca66-4ae0-a0f7-d7f48ea1a5dc admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:24:34.357160 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1246/2469] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jul 14 09:24:34 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 09:24:34.361143 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbebc42-ca66-4ae0-a0f7-d7f48ea1a5dc admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:24:34.370839 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbebc42-ca66-4ae0-a0f7-d7f48ea1a5dc admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:24:34.375647 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74713407-6bc5-471c-93dd-9940bda45813 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:24:34.375800 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1224/2470] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:24:34 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:24:34.379810 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74713407-6bc5-471c-93dd-9940bda45813 admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:24:34.380194 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74713407-6bc5-471c-93dd-9940bda45813 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:24:34.380478 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74713407-6bc5-471c-93dd-9940bda45813 admin admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:24:34.381215 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74713407-6bc5-471c-93dd-9940bda45813 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:24:34.382816 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74713407-6bc5-471c-93dd-9940bda45813 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:24:34.386501 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee6f6ef-1545-4aeb-b99d-529a497290a0 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:24:34.387889 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-fee6f6ef-1545-4aeb-b99d-529a497290a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:24:34.395402 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1247/2471] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jul 14 09:24:34 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 09:24:34.398294 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74713407-6bc5-471c-93dd-9940bda45813 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:24:34.407898 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74713407-6bc5-471c-93dd-9940bda45813 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:24:34.412772 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e088f0e-efff-4731-bf08-7575dacc8c35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:24:34.413650 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e088f0e-efff-4731-bf08-7575dacc8c35 admin admin] REQUEST_METHOD: `PUT` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:24:34.413929 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e088f0e-efff-4731-bf08-7575dacc8c35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:24:34.414199 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e088f0e-efff-4731-bf08-7575dacc8c35 admin admin] PATH_INFO: `/v3/projects/6ce22944330048629708fc174f53ecf0/users/fcf0b8fe011645a086044d84f6a34536/roles/d1691322ce6a4edab7ce722edf0f2988` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:24:34.429649 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e088f0e-efff-4731-bf08-7575dacc8c35 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ce22944330048629708fc174f53ecf0, user_id=fcf0b8fe011645a086044d84f6a34536, role_id=d1691322ce6a4edab7ce722edf0f2988)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:24:34.432352 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e088f0e-efff-4731-bf08-7575dacc8c35 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:24:34.446276 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee6f6ef-1545-4aeb-b99d-529a497290a0 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:24:34.456620 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1248/2472] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jul 14 09:24:34 2022] PUT /identity/v3/projects/6ce22944330048629708fc174f53ecf0/users/fcf0b8fe011645a086044d84f6a34536/roles/d1691322ce6a4edab7ce722edf0f2988 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 09:24:34.458838 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af774d34-42a1-46a0-bd52-eaa0b419eef6 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:24:34.459537 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af774d34-42a1-46a0-bd52-eaa0b419eef6 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:24:34.459537 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af774d34-42a1-46a0-bd52-eaa0b419eef6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:24:34.466101 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:24:34.487248 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-af774d34-42a1-46a0-bd52-eaa0b419eef6 None None] MFA Rules not processed for user `fcf0b8fe011645a086044d84f6a34536`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:24:34.488469 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-af774d34-42a1-46a0-bd52-eaa0b419eef6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:24:34.497095 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7638f7b-5710-4b1d-965a-aa4a6864c9df service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e76f1952a5b146808844b9f2b7395035', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:24:34.497751 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7638f7b-5710-4b1d-965a-aa4a6864c9df service cinder] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:24:34.498036 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7638f7b-5710-4b1d-965a-aa4a6864c9df service cinder] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:24:34.498314 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7638f7b-5710-4b1d-965a-aa4a6864c9df service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:24:34.502584 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:24:34.555298 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1249/2473] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:24:34 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 09:24:34.566213 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af774d34-42a1-46a0-bd52-eaa0b419eef6 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:24:34.568022 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-af774d34-42a1-46a0-bd52-eaa0b419eef6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:24:34.581545 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7638f7b-5710-4b1d-965a-aa4a6864c9df service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:24:34.583002 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7638f7b-5710-4b1d-965a-aa4a6864c9df service cinder] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:24:34.593372 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1225/2474] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:24:34 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 09:24:34.663498 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af774d34-42a1-46a0-bd52-eaa0b419eef6 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:24:34.669223 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58945f2c-f463-4b29-8cfa-e2c319abc961 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '538fff3c5a2b487eb44ae7c1a1198102', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:24:34.669888 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58945f2c-f463-4b29-8cfa-e2c319abc961 service nova] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:24:34.670170 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58945f2c-f463-4b29-8cfa-e2c319abc961 service nova] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:24:34.670447 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58945f2c-f463-4b29-8cfa-e2c319abc961 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:24:34.679016 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58945f2c-f463-4b29-8cfa-e2c319abc961 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:24:34.680789 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58945f2c-f463-4b29-8cfa-e2c319abc961 service nova] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:24:34.690266 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1250/2475] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:24:34 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:24:44.589693 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=25 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:24:44.686065 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=64 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:24:46.330422 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7638f7b-5710-4b1d-965a-aa4a6864c9df service cinder] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:24:46.390289 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7638f7b-5710-4b1d-965a-aa4a6864c9df service cinder] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:24:46.394833 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e997d7-c2e2-454d-991c-8ba5599ef257 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:24:46.396321 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e997d7-c2e2-454d-991c-8ba5599ef257 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:24:46.399533 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e997d7-c2e2-454d-991c-8ba5599ef257 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:24:46.399848 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e997d7-c2e2-454d-991c-8ba5599ef257 admin admin] PATH_INFO: `/v3/users/fcf0b8fe011645a086044d84f6a34536` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:24:46.401167 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e997d7-c2e2-454d-991c-8ba5599ef257 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fcf0b8fe011645a086044d84f6a34536)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:24:46.403266 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e997d7-c2e2-454d-991c-8ba5599ef257 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:24:46.456027 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-13e997d7-c2e2-454d-991c-8ba5599ef257 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fcf0b8fe011645a086044d84f6a34536'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:24:46.467476 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-13e997d7-c2e2-454d-991c-8ba5599ef257 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fcf0b8fe011645a086044d84f6a34536'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:24:46.468505 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-13e997d7-c2e2-454d-991c-8ba5599ef257 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fcf0b8fe011645a086044d84f6a34536'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:24:46.469178 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-13e997d7-c2e2-454d-991c-8ba5599ef257 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fcf0b8fe011645a086044d84f6a34536'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:24:46.478986 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-13e997d7-c2e2-454d-991c-8ba5599ef257 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fcf0b8fe011645a086044d84f6a34536'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:24:46.501032 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1226/2476] 10.222.0.26 () {58 vars in 1223 bytes} [Thu Jul 14 09:24:46 2022] DELETE /identity/v3/users/fcf0b8fe011645a086044d84f6a34536 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:24:46.505325 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c20397-b65d-4d93-ac49-c0d13d7746e8 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:24:46.505844 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c20397-b65d-4d93-ac49-c0d13d7746e8 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:24:46.505950 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c20397-b65d-4d93-ac49-c0d13d7746e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:24:46.524561 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-d5c20397-b65d-4d93-ac49-c0d13d7746e8 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:24:46.525744 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-d5c20397-b65d-4d93-ac49-c0d13d7746e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:24:46.607338 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1251/2477] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:24:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:24:46.617053 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e997d7-c2e2-454d-991c-8ba5599ef257 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:24:46.618489 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-13e997d7-c2e2-454d-991c-8ba5599ef257 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:24:46.678876 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e997d7-c2e2-454d-991c-8ba5599ef257 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:24:46.683648 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7596dcfa-cc3a-4655-acc6-e67c126dc384 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '952911c492444ffea4b211a24b564117', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:24:46.684293 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7596dcfa-cc3a-4655-acc6-e67c126dc384 service neutron] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:24:46.684581 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7596dcfa-cc3a-4655-acc6-e67c126dc384 service neutron] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:24:46.684863 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7596dcfa-cc3a-4655-acc6-e67c126dc384 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:24:46.697919 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7596dcfa-cc3a-4655-acc6-e67c126dc384 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:24:46.699497 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7596dcfa-cc3a-4655-acc6-e67c126dc384 service neutron] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:24:46.713228 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1227/2478] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:24:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:24:46.988335 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c20397-b65d-4d93-ac49-c0d13d7746e8 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:24:46.989774 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-d5c20397-b65d-4d93-ac49-c0d13d7746e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:24:47.002484 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c20397-b65d-4d93-ac49-c0d13d7746e8 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:24:47.007709 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d7b802-0246-4695-bc66-064feb17eaf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:24:47.008605 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d7b802-0246-4695-bc66-064feb17eaf3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:24:47.008885 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d7b802-0246-4695-bc66-064feb17eaf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:24:47.009253 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d7b802-0246-4695-bc66-064feb17eaf3 admin admin] PATH_INFO: `/v3/projects/6ce22944330048629708fc174f53ecf0` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:24:47.010519 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d7b802-0246-4695-bc66-064feb17eaf3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6ce22944330048629708fc174f53ecf0)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:24:47.012585 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d7b802-0246-4695-bc66-064feb17eaf3 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:24:47.073602 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-85d7b802-0246-4695-bc66-064feb17eaf3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6ce22944330048629708fc174f53ecf0', 'request_id': 'req-85d7b802-0246-4695-bc66-064feb17eaf3'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:24:47.080950 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-85d7b802-0246-4695-bc66-064feb17eaf3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6ce22944330048629708fc174f53ecf0', 'request_id': 'req-85d7b802-0246-4695-bc66-064feb17eaf3'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:24:47.086858 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-85d7b802-0246-4695-bc66-064feb17eaf3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6ce22944330048629708fc174f53ecf0', 'request_id': 'req-85d7b802-0246-4695-bc66-064feb17eaf3'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:24:47.100167 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-85d7b802-0246-4695-bc66-064feb17eaf3 admin admin] The token cache is being invalidate because project 6ce22944330048629708fc174f53ecf0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80382) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 09:24:47.100466 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-85d7b802-0246-4695-bc66-064feb17eaf3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:24:47.101076 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-85d7b802-0246-4695-bc66-064feb17eaf3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:24:47.102622 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1252/2479] 10.222.0.26 () {58 vars in 1232 bytes} [Thu Jul 14 09:24:46 2022] DELETE /identity/v3/projects/6ce22944330048629708fc174f53ecf0 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:24:47.107867 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fe8db3-6783-4f63-95a8-1424cace598b None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:24:47.108236 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fe8db3-6783-4f63-95a8-1424cace598b None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:24:47.108624 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fe8db3-6783-4f63-95a8-1424cace598b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:24:47.130520 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-e5fe8db3-6783-4f63-95a8-1424cace598b None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:24:47.131985 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-e5fe8db3-6783-4f63-95a8-1424cace598b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:24:47.152153 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:24:47.211113 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1228/2480] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:24:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:24:47.220678 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38400fe8-900e-4a06-89dc-f21d2d2d2a03 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:24:47.220967 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38400fe8-900e-4a06-89dc-f21d2d2d2a03 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:24:47.221253 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38400fe8-900e-4a06-89dc-f21d2d2d2a03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:24:47.241120 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-38400fe8-900e-4a06-89dc-f21d2d2d2a03 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:24:47.242408 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-38400fe8-900e-4a06-89dc-f21d2d2d2a03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:24:47.250849 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1253/2481] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:24:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:24:47.255224 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fe8db3-6783-4f63-95a8-1424cace598b None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:24:47.265494 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fe8db3-6783-4f63-95a8-1424cace598b None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:24:47.270878 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f0cb07-fdf6-42de-b839-c605cf2a9387 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:24:47.271577 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f0cb07-fdf6-42de-b839-c605cf2a9387 admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:24:47.271884 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f0cb07-fdf6-42de-b839-c605cf2a9387 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:24:47.272145 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f0cb07-fdf6-42de-b839-c605cf2a9387 admin admin] PATH_INFO: `/v3/domains` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:24:47.272918 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f0cb07-fdf6-42de-b839-c605cf2a9387 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:24:47.273222 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f0cb07-fdf6-42de-b839-c605cf2a9387 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:24:47.274880 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f0cb07-fdf6-42de-b839-c605cf2a9387 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:24:47.287248 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1229/2482] 10.222.0.26 () {58 vars in 1152 bytes} [Thu Jul 14 09:24:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:24:47.291884 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38400fe8-900e-4a06-89dc-f21d2d2d2a03 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:24:47.301512 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38400fe8-900e-4a06-89dc-f21d2d2d2a03 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:24:47.307109 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ca31d3-f5fb-4d2b-bd55-c7584029aff3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:24:47.307865 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ca31d3-f5fb-4d2b-bd55-c7584029aff3 admin admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:24:47.308181 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ca31d3-f5fb-4d2b-bd55-c7584029aff3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:24:47.308462 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ca31d3-f5fb-4d2b-bd55-c7584029aff3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:24:47.309179 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ca31d3-f5fb-4d2b-bd55-c7584029aff3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:24:47.311159 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ca31d3-f5fb-4d2b-bd55-c7584029aff3 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:24:47.342408 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1254/2483] 10.222.0.26 () {60 vars in 1152 bytes} [Thu Jul 14 09:24:47 2022] POST /identity/v3/projects => generated 371 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 09:24:47.347016 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f0cb07-fdf6-42de-b839-c605cf2a9387 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:24:47.358763 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f0cb07-fdf6-42de-b839-c605cf2a9387 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:24:47.421722 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2052bf-5530-4f57-9b99-c5a56c377633 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:24:47.422405 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2052bf-5530-4f57-9b99-c5a56c377633 admin admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:24:47.422689 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2052bf-5530-4f57-9b99-c5a56c377633 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:24:47.422965 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2052bf-5530-4f57-9b99-c5a56c377633 admin admin] PATH_INFO: `/v3/users` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:24:47.423702 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2052bf-5530-4f57-9b99-c5a56c377633 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:24:47.425860 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2052bf-5530-4f57-9b99-c5a56c377633 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:24:47.449401 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1230/2484] 10.222.0.26 () {60 vars in 1143 bytes} [Thu Jul 14 09:24:47 2022] POST /identity/v3/users => generated 286 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 09:24:47.453845 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ca31d3-f5fb-4d2b-bd55-c7584029aff3 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:24:47.468476 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ca31d3-f5fb-4d2b-bd55-c7584029aff3 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:24:47.474579 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b22aff-dc54-4489-8cca-10e895f8e9b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:24:47.475297 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b22aff-dc54-4489-8cca-10e895f8e9b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:24:47.475567 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b22aff-dc54-4489-8cca-10e895f8e9b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:24:47.475826 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b22aff-dc54-4489-8cca-10e895f8e9b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:24:47.476564 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b22aff-dc54-4489-8cca-10e895f8e9b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:24:47.478182 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b22aff-dc54-4489-8cca-10e895f8e9b9 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:24:47.489599 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1255/2485] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jul 14 09:24:47 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 09:24:47.493872 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2052bf-5530-4f57-9b99-c5a56c377633 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:24:47.504658 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2052bf-5530-4f57-9b99-c5a56c377633 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:24:47.509914 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e45b20-467e-4f70-9282-c8c5693f64f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:24:47.510578 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e45b20-467e-4f70-9282-c8c5693f64f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:24:47.510887 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e45b20-467e-4f70-9282-c8c5693f64f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:24:47.511168 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e45b20-467e-4f70-9282-c8c5693f64f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:24:47.511935 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e45b20-467e-4f70-9282-c8c5693f64f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:24:47.513571 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e45b20-467e-4f70-9282-c8c5693f64f6 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:24:47.525029 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1231/2486] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jul 14 09:24:47 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 09:24:47.529174 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b22aff-dc54-4489-8cca-10e895f8e9b9 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:24:47.539312 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b22aff-dc54-4489-8cca-10e895f8e9b9 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:24:47.544165 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1254c8cc-4420-42bb-ae67-e05cbaa01720 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:24:47.545052 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1254c8cc-4420-42bb-ae67-e05cbaa01720 admin admin] REQUEST_METHOD: `PUT` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:24:47.545327 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1254c8cc-4420-42bb-ae67-e05cbaa01720 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:24:47.545597 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1254c8cc-4420-42bb-ae67-e05cbaa01720 admin admin] PATH_INFO: `/v3/projects/14cc6a52e86d4e82b1daecae53ef98f1/users/990adcd3786543fa95a90eed65cace03/roles/d1691322ce6a4edab7ce722edf0f2988` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:24:47.560699 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1254c8cc-4420-42bb-ae67-e05cbaa01720 admin admin] RBAC: Authorizing `identity:create_grant(project_id=14cc6a52e86d4e82b1daecae53ef98f1, user_id=990adcd3786543fa95a90eed65cace03, role_id=d1691322ce6a4edab7ce722edf0f2988)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:24:47.563395 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1254c8cc-4420-42bb-ae67-e05cbaa01720 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:24:47.577474 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1256/2487] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jul 14 09:24:47 2022] PUT /identity/v3/projects/14cc6a52e86d4e82b1daecae53ef98f1/users/990adcd3786543fa95a90eed65cace03/roles/d1691322ce6a4edab7ce722edf0f2988 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 09:24:47.589655 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34cf4c4-53d6-4df0-9822-59cbe1913404 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:24:47.590006 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34cf4c4-53d6-4df0-9822-59cbe1913404 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:24:47.590327 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34cf4c4-53d6-4df0-9822-59cbe1913404 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:24:47.628086 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-d34cf4c4-53d6-4df0-9822-59cbe1913404 None None] MFA Rules not processed for user `990adcd3786543fa95a90eed65cace03`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:24:47.629314 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-d34cf4c4-53d6-4df0-9822-59cbe1913404 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:24:47.700559 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1232/2488] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:24:47 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:24:47.711660 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1254c8cc-4420-42bb-ae67-e05cbaa01720 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:24:47.713323 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-1254c8cc-4420-42bb-ae67-e05cbaa01720 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:24:47.735815 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:24:47.790739 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1254c8cc-4420-42bb-ae67-e05cbaa01720 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:24:47.797603 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d249e8-0bb2-47e3-b079-05659649518d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '538fff3c5a2b487eb44ae7c1a1198102', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:24:47.798321 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d249e8-0bb2-47e3-b079-05659649518d service nova] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:24:47.798589 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d249e8-0bb2-47e3-b079-05659649518d service nova] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:24:47.798907 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d249e8-0bb2-47e3-b079-05659649518d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:24:47.807161 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d249e8-0bb2-47e3-b079-05659649518d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:24:47.808921 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d249e8-0bb2-47e3-b079-05659649518d service nova] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:24:47.818170 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1257/2489] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:24:47 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:24:50.444922 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34cf4c4-53d6-4df0-9822-59cbe1913404 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:24:50.446894 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-d34cf4c4-53d6-4df0-9822-59cbe1913404 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:24:50.468166 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:24:50.517368 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d249e8-0bb2-47e3-b079-05659649518d service nova] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:24:50.517832 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34cf4c4-53d6-4df0-9822-59cbe1913404 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:24:50.524259 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27546a9-c249-4984-a2a4-b9dc0f071095 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:24:50.525173 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27546a9-c249-4984-a2a4-b9dc0f071095 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:24:50.525449 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27546a9-c249-4984-a2a4-b9dc0f071095 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:24:50.525720 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27546a9-c249-4984-a2a4-b9dc0f071095 admin admin] PATH_INFO: `/v3/users/f38291f7d1024c08b364e223ebf686f6` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:24:50.526928 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27546a9-c249-4984-a2a4-b9dc0f071095 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f38291f7d1024c08b364e223ebf686f6)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:24:50.527277 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d249e8-0bb2-47e3-b079-05659649518d service nova] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:24:50.529123 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27546a9-c249-4984-a2a4-b9dc0f071095 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:24:50.533863 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a62beb-8ada-4bcb-9082-bf2a1b99ca85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:24:50.534859 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a62beb-8ada-4bcb-9082-bf2a1b99ca85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:24:50.535230 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a62beb-8ada-4bcb-9082-bf2a1b99ca85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:24:50.535579 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a62beb-8ada-4bcb-9082-bf2a1b99ca85 admin admin] PATH_INFO: `/v3/users/990adcd3786543fa95a90eed65cace03` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:24:50.536916 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a62beb-8ada-4bcb-9082-bf2a1b99ca85 admin admin] RBAC: Authorizing `identity:delete_user(user_id=990adcd3786543fa95a90eed65cace03)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:24:50.538916 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a62beb-8ada-4bcb-9082-bf2a1b99ca85 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:24:50.580989 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-c27546a9-c249-4984-a2a4-b9dc0f071095 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f38291f7d1024c08b364e223ebf686f6'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:24:50.588800 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-c27546a9-c249-4984-a2a4-b9dc0f071095 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f38291f7d1024c08b364e223ebf686f6'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:24:50.589552 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-c27546a9-c249-4984-a2a4-b9dc0f071095 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f38291f7d1024c08b364e223ebf686f6'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:24:50.590195 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-c27546a9-c249-4984-a2a4-b9dc0f071095 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f38291f7d1024c08b364e223ebf686f6'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:24:50.591302 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-80a62beb-8ada-4bcb-9082-bf2a1b99ca85 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '990adcd3786543fa95a90eed65cace03'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:24:50.591928 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-80a62beb-8ada-4bcb-9082-bf2a1b99ca85 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '990adcd3786543fa95a90eed65cace03'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:24:50.597315 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-c27546a9-c249-4984-a2a4-b9dc0f071095 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f38291f7d1024c08b364e223ebf686f6'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:24:50.603831 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-80a62beb-8ada-4bcb-9082-bf2a1b99ca85 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '990adcd3786543fa95a90eed65cace03'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:24:50.612271 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-80a62beb-8ada-4bcb-9082-bf2a1b99ca85 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '990adcd3786543fa95a90eed65cace03'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:24:50.613044 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-80a62beb-8ada-4bcb-9082-bf2a1b99ca85 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '990adcd3786543fa95a90eed65cace03'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:24:50.615801 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1233/2490] 10.222.0.26 () {58 vars in 1223 bytes} [Thu Jul 14 09:24:50 2022] DELETE /identity/v3/users/f38291f7d1024c08b364e223ebf686f6 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:24:50.620277 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3570f33-791c-402a-8347-c32235c0a3d3 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:24:50.620794 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3570f33-791c-402a-8347-c32235c0a3d3 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:24:50.621235 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3570f33-791c-402a-8347-c32235c0a3d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:24:50.629468 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1258/2491] 10.222.0.26 () {58 vars in 1223 bytes} [Thu Jul 14 09:24:50 2022] DELETE /identity/v3/users/990adcd3786543fa95a90eed65cace03 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:24:50.634338 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d9b3bd-7b93-4041-954c-b3015673d46e None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:24:50.634774 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d9b3bd-7b93-4041-954c-b3015673d46e None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:24:50.635233 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d9b3bd-7b93-4041-954c-b3015673d46e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:24:50.638744 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-b3570f33-791c-402a-8347-c32235c0a3d3 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:24:50.639967 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-b3570f33-791c-402a-8347-c32235c0a3d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:24:50.653737 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-b7d9b3bd-7b93-4041-954c-b3015673d46e None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:24:50.654907 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-b7d9b3bd-7b93-4041-954c-b3015673d46e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:24:50.700813 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1234/2492] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:24:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 09:24:50.710565 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3570f33-791c-402a-8347-c32235c0a3d3 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:24:50.712005 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-b3570f33-791c-402a-8347-c32235c0a3d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:24:50.716959 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1259/2493] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:24:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 09:24:50.726167 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d9b3bd-7b93-4041-954c-b3015673d46e None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:24:50.727527 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-b7d9b3bd-7b93-4041-954c-b3015673d46e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:24:50.747121 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:24:50.752742 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:24:50.769844 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3570f33-791c-402a-8347-c32235c0a3d3 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:24:50.773813 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8577f1-9674-474f-815d-4bce0e142b51 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '952911c492444ffea4b211a24b564117', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:24:50.774472 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8577f1-9674-474f-815d-4bce0e142b51 service neutron] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:24:50.774760 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8577f1-9674-474f-815d-4bce0e142b51 service neutron] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:24:50.775040 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8577f1-9674-474f-815d-4bce0e142b51 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:24:50.784366 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8577f1-9674-474f-815d-4bce0e142b51 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:24:50.785799 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8577f1-9674-474f-815d-4bce0e142b51 service neutron] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:24:50.787886 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d9b3bd-7b93-4041-954c-b3015673d46e None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:24:50.791522 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ac873e-d1ac-49bd-91c6-a110c441adbc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '952911c492444ffea4b211a24b564117', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:24:50.792147 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ac873e-d1ac-49bd-91c6-a110c441adbc service neutron] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:24:50.792413 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ac873e-d1ac-49bd-91c6-a110c441adbc service neutron] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:24:50.792866 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ac873e-d1ac-49bd-91c6-a110c441adbc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:24:50.795785 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1235/2494] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:24:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 09:24:50.802272 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ac873e-d1ac-49bd-91c6-a110c441adbc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:24:50.803831 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ac873e-d1ac-49bd-91c6-a110c441adbc service neutron] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:24:50.814079 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1260/2495] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:24:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 09:24:51.201021 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8577f1-9674-474f-815d-4bce0e142b51 service neutron] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:24:51.202832 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-2f8577f1-9674-474f-815d-4bce0e142b51 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:24:51.212113 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ac873e-d1ac-49bd-91c6-a110c441adbc service neutron] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:24:51.213693 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-b5ac873e-d1ac-49bd-91c6-a110c441adbc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:24:51.226599 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8577f1-9674-474f-815d-4bce0e142b51 service neutron] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:24:51.228632 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ac873e-d1ac-49bd-91c6-a110c441adbc service neutron] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:24:51.233882 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8be4684-1aae-46f5-9a24-b95daa178213 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:24:51.237759 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8be4684-1aae-46f5-9a24-b95daa178213 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:24:51.237759 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8be4684-1aae-46f5-9a24-b95daa178213 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:24:51.237759 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8be4684-1aae-46f5-9a24-b95daa178213 admin admin] PATH_INFO: `/v3/projects/14cc6a52e86d4e82b1daecae53ef98f1` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:24:51.237759 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8be4684-1aae-46f5-9a24-b95daa178213 admin admin] RBAC: Authorizing `identity:delete_project(project_id=14cc6a52e86d4e82b1daecae53ef98f1)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:24:51.238905 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8be4684-1aae-46f5-9a24-b95daa178213 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:24:51.239371 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2844fcf-92c3-4da6-9d47-8e0f1ef06f8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:24:51.240458 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2844fcf-92c3-4da6-9d47-8e0f1ef06f8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:24:51.240758 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2844fcf-92c3-4da6-9d47-8e0f1ef06f8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:24:51.241094 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2844fcf-92c3-4da6-9d47-8e0f1ef06f8c admin admin] PATH_INFO: `/v3/projects/6cb67ef188a244779657a2c465ab4fe2` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:24:51.242556 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2844fcf-92c3-4da6-9d47-8e0f1ef06f8c admin admin] RBAC: Authorizing `identity:delete_project(project_id=6cb67ef188a244779657a2c465ab4fe2)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:24:51.244820 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2844fcf-92c3-4da6-9d47-8e0f1ef06f8c admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:24:51.300988 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-c8be4684-1aae-46f5-9a24-b95daa178213 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '14cc6a52e86d4e82b1daecae53ef98f1', 'request_id': 'req-c8be4684-1aae-46f5-9a24-b95daa178213'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:24:51.308430 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-c8be4684-1aae-46f5-9a24-b95daa178213 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '14cc6a52e86d4e82b1daecae53ef98f1', 'request_id': 'req-c8be4684-1aae-46f5-9a24-b95daa178213'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:24:51.315329 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-c8be4684-1aae-46f5-9a24-b95daa178213 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '14cc6a52e86d4e82b1daecae53ef98f1', 'request_id': 'req-c8be4684-1aae-46f5-9a24-b95daa178213'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:24:51.318067 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-c2844fcf-92c3-4da6-9d47-8e0f1ef06f8c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6cb67ef188a244779657a2c465ab4fe2', 'request_id': 'req-c2844fcf-92c3-4da6-9d47-8e0f1ef06f8c'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:24:51.325300 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-c2844fcf-92c3-4da6-9d47-8e0f1ef06f8c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6cb67ef188a244779657a2c465ab4fe2', 'request_id': 'req-c2844fcf-92c3-4da6-9d47-8e0f1ef06f8c'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:24:51.329513 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-c8be4684-1aae-46f5-9a24-b95daa178213 admin admin] The token cache is being invalidate because project 14cc6a52e86d4e82b1daecae53ef98f1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80382) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 09:24:51.329799 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-c8be4684-1aae-46f5-9a24-b95daa178213 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:24:51.330429 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-c8be4684-1aae-46f5-9a24-b95daa178213 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:24:51.332278 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1261/2496] 10.222.0.26 () {58 vars in 1232 bytes} [Thu Jul 14 09:24:51 2022] DELETE /identity/v3/projects/14cc6a52e86d4e82b1daecae53ef98f1 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:24:51.332441 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-c2844fcf-92c3-4da6-9d47-8e0f1ef06f8c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6cb67ef188a244779657a2c465ab4fe2', 'request_id': 'req-c2844fcf-92c3-4da6-9d47-8e0f1ef06f8c'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:24:51.337728 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753c8dcf-0176-4163-a015-12235bcb26c6 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:24:51.338045 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753c8dcf-0176-4163-a015-12235bcb26c6 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:24:51.338325 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753c8dcf-0176-4163-a015-12235bcb26c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:24:51.348273 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-c2844fcf-92c3-4da6-9d47-8e0f1ef06f8c admin admin] The token cache is being invalidate because project 6cb67ef188a244779657a2c465ab4fe2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80381) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 09:24:51.348612 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-c2844fcf-92c3-4da6-9d47-8e0f1ef06f8c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:24:51.349255 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-c2844fcf-92c3-4da6-9d47-8e0f1ef06f8c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:24:51.350895 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1236/2497] 10.222.0.26 () {58 vars in 1232 bytes} [Thu Jul 14 09:24:51 2022] DELETE /identity/v3/projects/6cb67ef188a244779657a2c465ab4fe2 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:24:51.355920 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118658aa-8095-40aa-8eb5-aee7f1105765 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:24:51.356329 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118658aa-8095-40aa-8eb5-aee7f1105765 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:24:51.356469 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118658aa-8095-40aa-8eb5-aee7f1105765 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:24:51.363572 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-753c8dcf-0176-4163-a015-12235bcb26c6 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:24:51.364819 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-753c8dcf-0176-4163-a015-12235bcb26c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:24:51.380117 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-118658aa-8095-40aa-8eb5-aee7f1105765 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:24:51.381291 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-118658aa-8095-40aa-8eb5-aee7f1105765 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:24:51.436747 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1262/2498] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:24:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 09:24:51.445875 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e746695b-0f69-4a6a-b044-dfe0c84e7d1e None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:24:51.446257 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e746695b-0f69-4a6a-b044-dfe0c84e7d1e None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:24:51.446447 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e746695b-0f69-4a6a-b044-dfe0c84e7d1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:24:51.456654 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1237/2499] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:24:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:24:51.464178 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06525a81-4f72-4345-9f3b-33f9d451e837 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:24:51.464509 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06525a81-4f72-4345-9f3b-33f9d451e837 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:24:51.464783 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06525a81-4f72-4345-9f3b-33f9d451e837 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:24:51.465121 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-e746695b-0f69-4a6a-b044-dfe0c84e7d1e None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:24:51.466325 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-e746695b-0f69-4a6a-b044-dfe0c84e7d1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:24:51.474627 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1263/2500] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:24:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 09:24:51.478839 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e746695b-0f69-4a6a-b044-dfe0c84e7d1e None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:24:51.482795 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-06525a81-4f72-4345-9f3b-33f9d451e837 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:24:51.484015 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-06525a81-4f72-4345-9f3b-33f9d451e837 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:24:51.488118 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e746695b-0f69-4a6a-b044-dfe0c84e7d1e None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:24:51.492763 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a58b901-a194-43e5-9675-f1f30d809fdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:24:51.493392 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a58b901-a194-43e5-9675-f1f30d809fdb admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:24:51.493673 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a58b901-a194-43e5-9675-f1f30d809fdb admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:24:51.493961 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a58b901-a194-43e5-9675-f1f30d809fdb admin admin] PATH_INFO: `/v3/domains` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:24:51.494673 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a58b901-a194-43e5-9675-f1f30d809fdb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:24:51.494975 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a58b901-a194-43e5-9675-f1f30d809fdb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:24:51.495339 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1238/2501] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:24:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 09:24:51.496361 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06525a81-4f72-4345-9f3b-33f9d451e837 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:24:51.496643 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a58b901-a194-43e5-9675-f1f30d809fdb admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:24:51.505685 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06525a81-4f72-4345-9f3b-33f9d451e837 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:24:51.509016 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1264/2502] 10.222.0.26 () {58 vars in 1152 bytes} [Thu Jul 14 09:24:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 09:24:51.513124 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a58b901-a194-43e5-9675-f1f30d809fdb admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:24:51.513737 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e602c8-a652-4463-bf96-80a5e04b35b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:24:51.514390 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e602c8-a652-4463-bf96-80a5e04b35b8 admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:24:51.514661 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e602c8-a652-4463-bf96-80a5e04b35b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:24:51.515608 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e602c8-a652-4463-bf96-80a5e04b35b8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:24:51.516350 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e602c8-a652-4463-bf96-80a5e04b35b8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:24:51.516653 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e602c8-a652-4463-bf96-80a5e04b35b8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:24:51.518270 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e602c8-a652-4463-bf96-80a5e04b35b8 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:24:51.523814 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a58b901-a194-43e5-9675-f1f30d809fdb admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:24:51.528769 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8341a198-bcc4-42b2-9a9d-892e68968d1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:24:51.534194 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8341a198-bcc4-42b2-9a9d-892e68968d1f admin admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:24:51.534194 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8341a198-bcc4-42b2-9a9d-892e68968d1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:24:51.534194 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8341a198-bcc4-42b2-9a9d-892e68968d1f admin admin] PATH_INFO: `/v3/projects` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:24:51.534194 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8341a198-bcc4-42b2-9a9d-892e68968d1f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:24:51.534194 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8341a198-bcc4-42b2-9a9d-892e68968d1f admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:24:51.534687 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1239/2503] 10.222.0.26 () {58 vars in 1152 bytes} [Thu Jul 14 09:24:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:24:51.536100 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e602c8-a652-4463-bf96-80a5e04b35b8 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:24:51.546732 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e602c8-a652-4463-bf96-80a5e04b35b8 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:24:51.556331 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cab0aba-8f2a-4e07-9230-5ce72bcd9136 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:24:51.557052 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cab0aba-8f2a-4e07-9230-5ce72bcd9136 admin admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:24:51.557318 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cab0aba-8f2a-4e07-9230-5ce72bcd9136 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:24:51.557596 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cab0aba-8f2a-4e07-9230-5ce72bcd9136 admin admin] PATH_INFO: `/v3/projects` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:24:51.558352 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cab0aba-8f2a-4e07-9230-5ce72bcd9136 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:24:51.560391 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cab0aba-8f2a-4e07-9230-5ce72bcd9136 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:24:51.562804 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1265/2504] 10.222.0.26 () {60 vars in 1152 bytes} [Thu Jul 14 09:24:51 2022] POST /identity/v3/projects => generated 403 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 09:24:51.566864 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8341a198-bcc4-42b2-9a9d-892e68968d1f admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:24:51.571917 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:24:51.591914 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1240/2505] 10.222.0.26 () {60 vars in 1152 bytes} [Thu Jul 14 09:24:51 2022] POST /identity/v3/projects => generated 357 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 09:24:51.593188 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cab0aba-8f2a-4e07-9230-5ce72bcd9136 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:24:51.618047 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:24:51.632985 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8341a198-bcc4-42b2-9a9d-892e68968d1f admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:24:51.658487 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cab0aba-8f2a-4e07-9230-5ce72bcd9136 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:24:51.662960 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51deb841-1abf-46c0-8835-858aad868016 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:24:51.663658 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51deb841-1abf-46c0-8835-858aad868016 admin admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:24:51.663944 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51deb841-1abf-46c0-8835-858aad868016 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:24:51.664222 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51deb841-1abf-46c0-8835-858aad868016 admin admin] PATH_INFO: `/v3/users` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:24:51.664910 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51deb841-1abf-46c0-8835-858aad868016 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:24:51.666870 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51deb841-1abf-46c0-8835-858aad868016 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:24:51.692701 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1241/2506] 10.222.0.26 () {60 vars in 1143 bytes} [Thu Jul 14 09:24:51 2022] POST /identity/v3/users => generated 279 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 09:24:51.692904 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df19fb21-cd14-4368-96ff-9895712bd0af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:24:51.692904 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df19fb21-cd14-4368-96ff-9895712bd0af admin admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:24:51.692904 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df19fb21-cd14-4368-96ff-9895712bd0af admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:24:51.692904 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df19fb21-cd14-4368-96ff-9895712bd0af admin admin] PATH_INFO: `/v3/users` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:24:51.693830 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51deb841-1abf-46c0-8835-858aad868016 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:24:51.694421 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df19fb21-cd14-4368-96ff-9895712bd0af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:24:51.696380 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df19fb21-cd14-4368-96ff-9895712bd0af admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:24:51.704461 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51deb841-1abf-46c0-8835-858aad868016 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:24:51.710601 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56f594e-b0b6-4999-ac3c-b77016e8472c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:24:51.711358 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56f594e-b0b6-4999-ac3c-b77016e8472c admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:24:51.711645 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56f594e-b0b6-4999-ac3c-b77016e8472c admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:24:51.711921 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56f594e-b0b6-4999-ac3c-b77016e8472c admin admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:24:51.712656 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56f594e-b0b6-4999-ac3c-b77016e8472c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:24:51.714297 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56f594e-b0b6-4999-ac3c-b77016e8472c admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:24:51.716851 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1266/2507] 10.222.0.26 () {60 vars in 1143 bytes} [Thu Jul 14 09:24:51 2022] POST /identity/v3/users => generated 302 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 09:24:51.721111 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df19fb21-cd14-4368-96ff-9895712bd0af admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:24:51.725555 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1242/2508] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jul 14 09:24:51 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 09:24:51.729388 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56f594e-b0b6-4999-ac3c-b77016e8472c admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:24:51.731293 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df19fb21-cd14-4368-96ff-9895712bd0af admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:24:51.736462 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b067ce-6864-4f7a-9fa4-8050299db418 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:24:51.737108 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b067ce-6864-4f7a-9fa4-8050299db418 admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:24:51.737405 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b067ce-6864-4f7a-9fa4-8050299db418 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:24:51.737678 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b067ce-6864-4f7a-9fa4-8050299db418 admin admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:24:51.738407 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b067ce-6864-4f7a-9fa4-8050299db418 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:24:51.739424 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56f594e-b0b6-4999-ac3c-b77016e8472c admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:24:51.740060 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b067ce-6864-4f7a-9fa4-8050299db418 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:24:51.744042 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c91c94-a90f-4d73-bc89-a873f90ae1f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:24:51.744684 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c91c94-a90f-4d73-bc89-a873f90ae1f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:24:51.744973 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c91c94-a90f-4d73-bc89-a873f90ae1f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:24:51.745698 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c91c94-a90f-4d73-bc89-a873f90ae1f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:24:51.746004 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c91c94-a90f-4d73-bc89-a873f90ae1f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:24:51.747737 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c91c94-a90f-4d73-bc89-a873f90ae1f1 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:24:51.750298 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1267/2509] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jul 14 09:24:51 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 09:24:51.754275 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b067ce-6864-4f7a-9fa4-8050299db418 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:24:51.758652 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1243/2510] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jul 14 09:24:51 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 09:24:51.762587 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c91c94-a90f-4d73-bc89-a873f90ae1f1 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:24:51.764868 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b067ce-6864-4f7a-9fa4-8050299db418 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:24:51.770799 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa49d6ea-fac6-4b38-b72c-23145021bdf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:24:51.771525 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa49d6ea-fac6-4b38-b72c-23145021bdf7 admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:24:51.771811 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa49d6ea-fac6-4b38-b72c-23145021bdf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:24:51.772213 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa49d6ea-fac6-4b38-b72c-23145021bdf7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:24:51.772736 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c91c94-a90f-4d73-bc89-a873f90ae1f1 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:24:51.772975 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa49d6ea-fac6-4b38-b72c-23145021bdf7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:24:51.774636 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa49d6ea-fac6-4b38-b72c-23145021bdf7 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:24:51.777428 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c86ae5-ebbb-4aae-aa77-6e002926713a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:24:51.778315 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c86ae5-ebbb-4aae-aa77-6e002926713a admin admin] REQUEST_METHOD: `PUT` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:24:51.778598 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c86ae5-ebbb-4aae-aa77-6e002926713a admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:24:51.778875 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c86ae5-ebbb-4aae-aa77-6e002926713a admin admin] PATH_INFO: `/v3/projects/65793779a11d41d893de762d062dc422/users/b5e7d732408140dcb3eba54d165b18ce/roles/d1691322ce6a4edab7ce722edf0f2988` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:24:51.785630 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1268/2511] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jul 14 09:24:51 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 09:24:51.789881 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa49d6ea-fac6-4b38-b72c-23145021bdf7 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:24:51.793903 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c86ae5-ebbb-4aae-aa77-6e002926713a admin admin] RBAC: Authorizing `identity:create_grant(project_id=65793779a11d41d893de762d062dc422, user_id=b5e7d732408140dcb3eba54d165b18ce, role_id=d1691322ce6a4edab7ce722edf0f2988)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:24:51.796607 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c86ae5-ebbb-4aae-aa77-6e002926713a admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:24:51.800152 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa49d6ea-fac6-4b38-b72c-23145021bdf7 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:24:51.805605 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0903fe-5c3a-44de-bcb6-a6c769c1b8a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:24:51.806491 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0903fe-5c3a-44de-bcb6-a6c769c1b8a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:24:51.806772 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0903fe-5c3a-44de-bcb6-a6c769c1b8a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:24:51.807045 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0903fe-5c3a-44de-bcb6-a6c769c1b8a5 admin admin] PATH_INFO: `/v3/projects/88c9e65cc4034d3d95d24b73ac06bd33/users/acb569ae043f4415810e587406116608/roles/d1691322ce6a4edab7ce722edf0f2988` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:24:51.810958 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1244/2512] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jul 14 09:24:51 2022] PUT /identity/v3/projects/65793779a11d41d893de762d062dc422/users/b5e7d732408140dcb3eba54d165b18ce/roles/d1691322ce6a4edab7ce722edf0f2988 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 09:24:51.822327 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0903fe-5c3a-44de-bcb6-a6c769c1b8a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=88c9e65cc4034d3d95d24b73ac06bd33, user_id=acb569ae043f4415810e587406116608, role_id=d1691322ce6a4edab7ce722edf0f2988)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:24:51.822918 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b748d276-2d83-4880-8cab-24670b6b4a08 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:24:51.823218 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b748d276-2d83-4880-8cab-24670b6b4a08 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:24:51.823541 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b748d276-2d83-4880-8cab-24670b6b4a08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:24:51.825090 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0903fe-5c3a-44de-bcb6-a6c769c1b8a5 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:24:51.838441 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1269/2513] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jul 14 09:24:51 2022] PUT /identity/v3/projects/88c9e65cc4034d3d95d24b73ac06bd33/users/acb569ae043f4415810e587406116608/roles/d1691322ce6a4edab7ce722edf0f2988 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 09:24:51.850779 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ae01e5-72f8-4ffa-87f3-c35360d5b46e None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:24:51.851064 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ae01e5-72f8-4ffa-87f3-c35360d5b46e None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:24:51.851379 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ae01e5-72f8-4ffa-87f3-c35360d5b46e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:24:51.856550 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-b748d276-2d83-4880-8cab-24670b6b4a08 None None] MFA Rules not processed for user `b5e7d732408140dcb3eba54d165b18ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:24:51.857755 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-b748d276-2d83-4880-8cab-24670b6b4a08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:24:51.877970 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-60ae01e5-72f8-4ffa-87f3-c35360d5b46e None None] MFA Rules not processed for user `acb569ae043f4415810e587406116608`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:24:51.879174 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-60ae01e5-72f8-4ffa-87f3-c35360d5b46e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:24:51.921047 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:24:51.921337 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1245/2514] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:24:51 2022] POST /identity/v3/auth/tokens => generated 3224 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:24:51.925971 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b748d276-2d83-4880-8cab-24670b6b4a08 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:24:51.948621 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1270/2515] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:24:51 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 09:24:51.952957 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ae01e5-72f8-4ffa-87f3-c35360d5b46e None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:24:51.983039 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:24:51.989983 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b748d276-2d83-4880-8cab-24670b6b4a08 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:24:51.995286 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0fcf79-1471-4f21-a099-e0d1d5de8b81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:24:51.996068 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0fcf79-1471-4f21-a099-e0d1d5de8b81 admin admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:24:51.996355 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0fcf79-1471-4f21-a099-e0d1d5de8b81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:24:51.996629 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0fcf79-1471-4f21-a099-e0d1d5de8b81 admin admin] PATH_INFO: `/v3/projects` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:24:51.997329 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0fcf79-1471-4f21-a099-e0d1d5de8b81 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:24:51.999265 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0fcf79-1471-4f21-a099-e0d1d5de8b81 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:24:52.016056 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ae01e5-72f8-4ffa-87f3-c35360d5b46e None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:24:52.021403 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a66f0e-4435-4b75-b8c0-dc40b0d7faa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:24:52.022061 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a66f0e-4435-4b75-b8c0-dc40b0d7faa9 admin admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:24:52.022343 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a66f0e-4435-4b75-b8c0-dc40b0d7faa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:24:52.022752 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a66f0e-4435-4b75-b8c0-dc40b0d7faa9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:24:52.023482 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a66f0e-4435-4b75-b8c0-dc40b0d7faa9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:24:52.024727 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1246/2516] 10.222.0.26 () {60 vars in 1152 bytes} [Thu Jul 14 09:24:51 2022] POST /identity/v3/projects => generated 355 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 09:24:52.025420 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a66f0e-4435-4b75-b8c0-dc40b0d7faa9 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:24:52.028903 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0fcf79-1471-4f21-a099-e0d1d5de8b81 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:24:52.052560 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1271/2517] 10.222.0.26 () {60 vars in 1152 bytes} [Thu Jul 14 09:24:51 2022] POST /identity/v3/projects => generated 403 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 09:24:52.057006 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a66f0e-4435-4b75-b8c0-dc40b0d7faa9 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:24:52.091012 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0fcf79-1471-4f21-a099-e0d1d5de8b81 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:24:52.118858 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a66f0e-4435-4b75-b8c0-dc40b0d7faa9 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:24:52.124562 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f76e8b-c19c-4f3f-a4f6-4cf4dc701826 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:24:52.125395 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f76e8b-c19c-4f3f-a4f6-4cf4dc701826 admin admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:24:52.125701 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f76e8b-c19c-4f3f-a4f6-4cf4dc701826 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:24:52.125959 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f76e8b-c19c-4f3f-a4f6-4cf4dc701826 admin admin] PATH_INFO: `/v3/users` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:24:52.126677 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f76e8b-c19c-4f3f-a4f6-4cf4dc701826 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:24:52.128734 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f76e8b-c19c-4f3f-a4f6-4cf4dc701826 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:24:52.154657 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1272/2518] 10.222.0.26 () {60 vars in 1143 bytes} [Thu Jul 14 09:24:52 2022] POST /identity/v3/users => generated 308 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 09:24:52.155879 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f76e8b-c19c-4f3f-a4f6-4cf4dc701826 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:24:52.166530 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f76e8b-c19c-4f3f-a4f6-4cf4dc701826 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:24:52.171509 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8245f8b6-d841-4f60-951a-de694016b4e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:24:52.172206 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8245f8b6-d841-4f60-951a-de694016b4e7 admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:24:52.172487 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8245f8b6-d841-4f60-951a-de694016b4e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:24:52.172783 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8245f8b6-d841-4f60-951a-de694016b4e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:24:52.173528 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8245f8b6-d841-4f60-951a-de694016b4e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:24:52.175234 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8245f8b6-d841-4f60-951a-de694016b4e7 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:24:52.189696 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1273/2519] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jul 14 09:24:52 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 09:24:52.191076 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8245f8b6-d841-4f60-951a-de694016b4e7 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:24:52.202087 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8245f8b6-d841-4f60-951a-de694016b4e7 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:24:52.207309 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69821a53-4b9a-4941-aafd-c8dc6a094806 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:24:52.208230 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69821a53-4b9a-4941-aafd-c8dc6a094806 admin admin] REQUEST_METHOD: `PUT` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:24:52.208514 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69821a53-4b9a-4941-aafd-c8dc6a094806 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:24:52.208778 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69821a53-4b9a-4941-aafd-c8dc6a094806 admin admin] PATH_INFO: `/v3/projects/792143568a794c458e1109cc41c6f947/users/8990965d35bb40c99d0c95595f06fb80/roles/7b549d58d1724395b343609d895c80d0` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:24:52.225115 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69821a53-4b9a-4941-aafd-c8dc6a094806 admin admin] RBAC: Authorizing `identity:create_grant(project_id=792143568a794c458e1109cc41c6f947, user_id=8990965d35bb40c99d0c95595f06fb80, role_id=7b549d58d1724395b343609d895c80d0)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:24:52.227846 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69821a53-4b9a-4941-aafd-c8dc6a094806 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:24:52.242932 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1274/2520] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jul 14 09:24:52 2022] PUT /identity/v3/projects/792143568a794c458e1109cc41c6f947/users/8990965d35bb40c99d0c95595f06fb80/roles/7b549d58d1724395b343609d895c80d0 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:24:52.247427 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69821a53-4b9a-4941-aafd-c8dc6a094806 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:24:52.277739 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:24:52.285347 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825ee2ab-d12b-434f-b3e0-7105a1eec617 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:24:52.286334 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825ee2ab-d12b-434f-b3e0-7105a1eec617 admin admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:24:52.286648 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825ee2ab-d12b-434f-b3e0-7105a1eec617 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:24:52.286976 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825ee2ab-d12b-434f-b3e0-7105a1eec617 admin admin] PATH_INFO: `/v3/users` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:24:52.287759 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825ee2ab-d12b-434f-b3e0-7105a1eec617 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:24:52.289848 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825ee2ab-d12b-434f-b3e0-7105a1eec617 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:24:52.314081 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1247/2521] 10.222.0.26 () {60 vars in 1143 bytes} [Thu Jul 14 09:24:52 2022] POST /identity/v3/users => generated 284 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 09:24:52.315123 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825ee2ab-d12b-434f-b3e0-7105a1eec617 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:24:52.319643 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69821a53-4b9a-4941-aafd-c8dc6a094806 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:24:52.325625 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f344cdc-20c3-4d2e-bdea-531788c6a562 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:24:52.326339 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f344cdc-20c3-4d2e-bdea-531788c6a562 admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:24:52.326629 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f344cdc-20c3-4d2e-bdea-531788c6a562 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:24:52.326740 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825ee2ab-d12b-434f-b3e0-7105a1eec617 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:24:52.326905 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f344cdc-20c3-4d2e-bdea-531788c6a562 admin admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:24:52.327715 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f344cdc-20c3-4d2e-bdea-531788c6a562 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:24:52.329286 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f344cdc-20c3-4d2e-bdea-531788c6a562 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:24:52.331471 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353c64e9-9464-40bb-9197-8792e897d82e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:24:52.332169 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353c64e9-9464-40bb-9197-8792e897d82e admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:24:52.332473 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353c64e9-9464-40bb-9197-8792e897d82e admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:24:52.332771 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353c64e9-9464-40bb-9197-8792e897d82e admin admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:24:52.333522 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353c64e9-9464-40bb-9197-8792e897d82e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:24:52.335146 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353c64e9-9464-40bb-9197-8792e897d82e admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:24:52.343036 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1275/2522] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jul 14 09:24:52 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 09:24:52.344052 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f344cdc-20c3-4d2e-bdea-531788c6a562 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:24:52.345444 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1248/2523] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jul 14 09:24:52 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 09:24:52.349265 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353c64e9-9464-40bb-9197-8792e897d82e admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:24:52.353782 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f344cdc-20c3-4d2e-bdea-531788c6a562 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:24:52.358800 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353c64e9-9464-40bb-9197-8792e897d82e admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:24:52.359182 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d03fe6-c188-4384-b8de-8fb567064f67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:24:52.360088 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d03fe6-c188-4384-b8de-8fb567064f67 admin admin] REQUEST_METHOD: `PUT` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:24:52.360370 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d03fe6-c188-4384-b8de-8fb567064f67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:24:52.360666 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d03fe6-c188-4384-b8de-8fb567064f67 admin admin] PATH_INFO: `/v3/projects/792143568a794c458e1109cc41c6f947/users/8990965d35bb40c99d0c95595f06fb80/roles/7b549d58d1724395b343609d895c80d0` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:24:52.362621 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d03fe6-c188-4384-b8de-8fb567064f67 admin admin] RBAC: Authorizing `identity:create_grant(project_id=792143568a794c458e1109cc41c6f947, user_id=8990965d35bb40c99d0c95595f06fb80, role_id=7b549d58d1724395b343609d895c80d0)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:24:52.363418 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f76475-e80c-49af-b717-530f17cf43f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:24:52.364311 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f76475-e80c-49af-b717-530f17cf43f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:24:52.364598 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f76475-e80c-49af-b717-530f17cf43f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:24:52.364875 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f76475-e80c-49af-b717-530f17cf43f8 admin admin] PATH_INFO: `/v3/projects/410d02e8ff244020ad902bf578383bc9/users/8d192883bb6c49d989a203a0004aa7bc/roles/7b549d58d1724395b343609d895c80d0` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:24:52.365360 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d03fe6-c188-4384-b8de-8fb567064f67 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:24:52.375778 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1276/2524] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jul 14 09:24:52 2022] PUT /identity/v3/projects/792143568a794c458e1109cc41c6f947/users/8990965d35bb40c99d0c95595f06fb80/roles/7b549d58d1724395b343609d895c80d0 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:24:52.379830 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f76475-e80c-49af-b717-530f17cf43f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=410d02e8ff244020ad902bf578383bc9, user_id=8d192883bb6c49d989a203a0004aa7bc, role_id=7b549d58d1724395b343609d895c80d0)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:24:52.382566 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f76475-e80c-49af-b717-530f17cf43f8 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:24:52.385637 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395a38cb-e53c-46b7-86fc-c4e8634a3e96 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:24:52.385894 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395a38cb-e53c-46b7-86fc-c4e8634a3e96 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:24:52.386214 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395a38cb-e53c-46b7-86fc-c4e8634a3e96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:24:52.399300 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1249/2525] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jul 14 09:24:52 2022] PUT /identity/v3/projects/410d02e8ff244020ad902bf578383bc9/users/8d192883bb6c49d989a203a0004aa7bc/roles/7b549d58d1724395b343609d895c80d0 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 09:24:52.403762 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f76475-e80c-49af-b717-530f17cf43f8 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:24:52.417583 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-395a38cb-e53c-46b7-86fc-c4e8634a3e96 None None] MFA Rules not processed for user `8990965d35bb40c99d0c95595f06fb80`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:24:52.418812 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-395a38cb-e53c-46b7-86fc-c4e8634a3e96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:24:52.449902 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:24:52.469742 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f76475-e80c-49af-b717-530f17cf43f8 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:24:52.474648 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d2599f-481a-4b84-9da4-5cafe9c455a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:24:52.475351 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d2599f-481a-4b84-9da4-5cafe9c455a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:24:52.475871 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d2599f-481a-4b84-9da4-5cafe9c455a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:24:52.475984 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d2599f-481a-4b84-9da4-5cafe9c455a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:24:52.476680 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d2599f-481a-4b84-9da4-5cafe9c455a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:24:52.478356 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d2599f-481a-4b84-9da4-5cafe9c455a9 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:24:52.489838 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1250/2526] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jul 14 09:24:52 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 09:24:52.493819 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d2599f-481a-4b84-9da4-5cafe9c455a9 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:24:52.504754 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d2599f-481a-4b84-9da4-5cafe9c455a9 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:24:52.509110 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cab59d6-9be9-431a-9387-369334637cd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:24:52.510096 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1277/2527] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:24:52 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 09:24:52.510221 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cab59d6-9be9-431a-9387-369334637cd9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:24:52.510316 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cab59d6-9be9-431a-9387-369334637cd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:24:52.510543 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cab59d6-9be9-431a-9387-369334637cd9 admin admin] PATH_INFO: `/v3/projects/410d02e8ff244020ad902bf578383bc9/users/8d192883bb6c49d989a203a0004aa7bc/roles/7b549d58d1724395b343609d895c80d0` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:24:52.512632 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cab59d6-9be9-431a-9387-369334637cd9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=410d02e8ff244020ad902bf578383bc9, user_id=8d192883bb6c49d989a203a0004aa7bc, role_id=7b549d58d1724395b343609d895c80d0)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:24:52.515733 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cab59d6-9be9-431a-9387-369334637cd9 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:24:52.521437 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395a38cb-e53c-46b7-86fc-c4e8634a3e96 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:24:52.522920 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-395a38cb-e53c-46b7-86fc-c4e8634a3e96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:24:52.536946 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1251/2528] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jul 14 09:24:52 2022] PUT /identity/v3/projects/410d02e8ff244020ad902bf578383bc9/users/8d192883bb6c49d989a203a0004aa7bc/roles/7b549d58d1724395b343609d895c80d0 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:24:52.538879 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6185aa-6e7c-41ff-ae37-26085b25e7e1 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:24:52.539159 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6185aa-6e7c-41ff-ae37-26085b25e7e1 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:24:52.539465 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6185aa-6e7c-41ff-ae37-26085b25e7e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:24:52.570171 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-2e6185aa-6e7c-41ff-ae37-26085b25e7e1 None None] MFA Rules not processed for user `8d192883bb6c49d989a203a0004aa7bc`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:24:52.571650 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-2e6185aa-6e7c-41ff-ae37-26085b25e7e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:24:52.583102 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395a38cb-e53c-46b7-86fc-c4e8634a3e96 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:24:52.637049 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:24:52.643348 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c66df58-cb80-4d7b-8eee-88ffbc54b302 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e76f1952a5b146808844b9f2b7395035', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:24:52.644142 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c66df58-cb80-4d7b-8eee-88ffbc54b302 service cinder] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:24:52.644614 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c66df58-cb80-4d7b-8eee-88ffbc54b302 service cinder] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:24:52.644905 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c66df58-cb80-4d7b-8eee-88ffbc54b302 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:24:52.650886 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1252/2529] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:24:52 2022] POST /identity/v3/auth/tokens => generated 3289 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:24:52.662427 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6185aa-6e7c-41ff-ae37-26085b25e7e1 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:24:52.663786 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-2e6185aa-6e7c-41ff-ae37-26085b25e7e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:24:52.675493 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6185aa-6e7c-41ff-ae37-26085b25e7e1 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:24:52.679426 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de0a9a2-a711-4565-af5c-8610f6745a7e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e76f1952a5b146808844b9f2b7395035', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:24:52.680083 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de0a9a2-a711-4565-af5c-8610f6745a7e service cinder] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:24:52.680363 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de0a9a2-a711-4565-af5c-8610f6745a7e service cinder] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:24:52.680638 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de0a9a2-a711-4565-af5c-8610f6745a7e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:24:52.689881 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de0a9a2-a711-4565-af5c-8610f6745a7e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:24:52.691369 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de0a9a2-a711-4565-af5c-8610f6745a7e service cinder] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:24:52.701789 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1253/2530] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:24:52 2022] GET /identity/v3/auth/tokens => generated 3289 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 09:24:52.706265 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c66df58-cb80-4d7b-8eee-88ffbc54b302 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:24:52.707788 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c66df58-cb80-4d7b-8eee-88ffbc54b302 service cinder] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:24:52.717561 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1278/2531] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:24:52 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 09:24:52.778127 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de0a9a2-a711-4565-af5c-8610f6745a7e service cinder] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:24:52.787525 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de0a9a2-a711-4565-af5c-8610f6745a7e service cinder] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:24:52.791065 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fced7173-1e99-4364-b289-a1100fbeba47 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e76f1952a5b146808844b9f2b7395035', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:24:52.791747 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fced7173-1e99-4364-b289-a1100fbeba47 service cinder] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:24:52.792037 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fced7173-1e99-4364-b289-a1100fbeba47 service cinder] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:24:52.792315 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fced7173-1e99-4364-b289-a1100fbeba47 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:24:52.854238 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fced7173-1e99-4364-b289-a1100fbeba47 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:24:52.855815 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fced7173-1e99-4364-b289-a1100fbeba47 service cinder] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:24:52.865564 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1254/2532] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:24:52 2022] GET /identity/v3/auth/tokens => generated 3224 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:24:52.873211 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c66df58-cb80-4d7b-8eee-88ffbc54b302 service cinder] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:24:52.940231 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c66df58-cb80-4d7b-8eee-88ffbc54b302 service cinder] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:24:52.944899 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de070c7-9265-463b-b986-f8f95517df86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:24:52.945787 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de070c7-9265-463b-b986-f8f95517df86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:24:52.946066 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de070c7-9265-463b-b986-f8f95517df86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:24:52.946339 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de070c7-9265-463b-b986-f8f95517df86 admin admin] PATH_INFO: `/v3/users/acb569ae043f4415810e587406116608` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:24:52.947681 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de070c7-9265-463b-b986-f8f95517df86 admin admin] RBAC: Authorizing `identity:delete_user(user_id=acb569ae043f4415810e587406116608)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:24:52.950029 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de070c7-9265-463b-b986-f8f95517df86 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:24:53.008691 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-7de070c7-9265-463b-b986-f8f95517df86 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'acb569ae043f4415810e587406116608'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:24:53.009327 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-7de070c7-9265-463b-b986-f8f95517df86 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'acb569ae043f4415810e587406116608'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:24:53.021797 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-7de070c7-9265-463b-b986-f8f95517df86 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'acb569ae043f4415810e587406116608'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:24:53.030006 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-7de070c7-9265-463b-b986-f8f95517df86 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'acb569ae043f4415810e587406116608'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:24:53.030631 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-7de070c7-9265-463b-b986-f8f95517df86 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'acb569ae043f4415810e587406116608'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:24:53.043166 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1279/2533] 10.222.0.26 () {58 vars in 1223 bytes} [Thu Jul 14 09:24:52 2022] DELETE /identity/v3/users/acb569ae043f4415810e587406116608 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:24:53.047661 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f0ee20-3a55-4de6-8ace-eeee2deb945e None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:24:53.047954 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f0ee20-3a55-4de6-8ace-eeee2deb945e None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:24:53.048235 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f0ee20-3a55-4de6-8ace-eeee2deb945e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:24:53.065331 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-f9f0ee20-3a55-4de6-8ace-eeee2deb945e None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:24:53.066548 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-f9f0ee20-3a55-4de6-8ace-eeee2deb945e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:24:53.096838 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:24:53.131145 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1255/2534] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:24:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:24:53.140872 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de070c7-9265-463b-b986-f8f95517df86 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:24:53.142308 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-7de070c7-9265-463b-b986-f8f95517df86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:24:53.165120 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:24:53.206205 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de070c7-9265-463b-b986-f8f95517df86 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:24:53.209963 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1eed536-73dd-46b6-ab12-82c3cb2419c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '952911c492444ffea4b211a24b564117', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:24:53.210610 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eed536-73dd-46b6-ab12-82c3cb2419c2 service neutron] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:24:53.210904 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eed536-73dd-46b6-ab12-82c3cb2419c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:24:53.211151 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eed536-73dd-46b6-ab12-82c3cb2419c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:24:53.220403 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1eed536-73dd-46b6-ab12-82c3cb2419c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:24:53.221847 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1eed536-73dd-46b6-ab12-82c3cb2419c2 service neutron] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:24:53.231579 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1280/2535] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:24:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:24:53.503866 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f0ee20-3a55-4de6-8ace-eeee2deb945e None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:24:53.505449 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-f9f0ee20-3a55-4de6-8ace-eeee2deb945e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:24:53.518428 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f0ee20-3a55-4de6-8ace-eeee2deb945e None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:24:53.524231 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633b5a1b-2eae-4a43-93f8-a68e361ee443 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:24:53.525214 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633b5a1b-2eae-4a43-93f8-a68e361ee443 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:24:53.525624 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633b5a1b-2eae-4a43-93f8-a68e361ee443 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:24:53.525953 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633b5a1b-2eae-4a43-93f8-a68e361ee443 admin admin] PATH_INFO: `/v3/projects/88c9e65cc4034d3d95d24b73ac06bd33` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:24:53.527626 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633b5a1b-2eae-4a43-93f8-a68e361ee443 admin admin] RBAC: Authorizing `identity:delete_project(project_id=88c9e65cc4034d3d95d24b73ac06bd33)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:24:53.529549 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633b5a1b-2eae-4a43-93f8-a68e361ee443 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:24:53.586993 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-633b5a1b-2eae-4a43-93f8-a68e361ee443 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '88c9e65cc4034d3d95d24b73ac06bd33', 'request_id': 'req-633b5a1b-2eae-4a43-93f8-a68e361ee443'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:24:53.593986 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-633b5a1b-2eae-4a43-93f8-a68e361ee443 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '88c9e65cc4034d3d95d24b73ac06bd33', 'request_id': 'req-633b5a1b-2eae-4a43-93f8-a68e361ee443'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:24:53.600432 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-633b5a1b-2eae-4a43-93f8-a68e361ee443 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '88c9e65cc4034d3d95d24b73ac06bd33', 'request_id': 'req-633b5a1b-2eae-4a43-93f8-a68e361ee443'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:24:53.613665 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-633b5a1b-2eae-4a43-93f8-a68e361ee443 admin admin] The token cache is being invalidate because project 88c9e65cc4034d3d95d24b73ac06bd33 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80381) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 09:24:53.614769 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-633b5a1b-2eae-4a43-93f8-a68e361ee443 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:24:53.615449 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-633b5a1b-2eae-4a43-93f8-a68e361ee443 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:24:53.617147 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1256/2536] 10.222.0.26 () {58 vars in 1232 bytes} [Thu Jul 14 09:24:53 2022] DELETE /identity/v3/projects/88c9e65cc4034d3d95d24b73ac06bd33 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:24:53.621424 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1eed536-73dd-46b6-ab12-82c3cb2419c2 service neutron] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:24:53.631084 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1eed536-73dd-46b6-ab12-82c3cb2419c2 service neutron] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:24:53.709640 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ec3748-a26c-4b3d-82f3-ee542a1a6c77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:24:53.710873 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ec3748-a26c-4b3d-82f3-ee542a1a6c77 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:24:53.712069 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ec3748-a26c-4b3d-82f3-ee542a1a6c77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:24:53.712542 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ec3748-a26c-4b3d-82f3-ee542a1a6c77 admin admin] PATH_INFO: `/v3/users/8990965d35bb40c99d0c95595f06fb80` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:24:53.714335 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ec3748-a26c-4b3d-82f3-ee542a1a6c77 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8990965d35bb40c99d0c95595f06fb80)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:24:53.717157 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ec3748-a26c-4b3d-82f3-ee542a1a6c77 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:24:53.785410 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-a2ec3748-a26c-4b3d-82f3-ee542a1a6c77 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8990965d35bb40c99d0c95595f06fb80'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:24:53.786079 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-a2ec3748-a26c-4b3d-82f3-ee542a1a6c77 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8990965d35bb40c99d0c95595f06fb80'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:24:53.797928 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-a2ec3748-a26c-4b3d-82f3-ee542a1a6c77 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8990965d35bb40c99d0c95595f06fb80'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:24:53.805032 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-a2ec3748-a26c-4b3d-82f3-ee542a1a6c77 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8990965d35bb40c99d0c95595f06fb80'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:24:53.805866 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-a2ec3748-a26c-4b3d-82f3-ee542a1a6c77 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8990965d35bb40c99d0c95595f06fb80'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:24:53.819222 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1281/2537] 10.222.0.26 () {58 vars in 1223 bytes} [Thu Jul 14 09:24:53 2022] DELETE /identity/v3/users/8990965d35bb40c99d0c95595f06fb80 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:24:54.100083 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633b5a1b-2eae-4a43-93f8-a68e361ee443 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:24:54.101485 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-633b5a1b-2eae-4a43-93f8-a68e361ee443 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:24:54.116648 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633b5a1b-2eae-4a43-93f8-a68e361ee443 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:24:54.207849 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302020db-0f2f-40f0-ac78-4c77dc29d24d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:24:54.208844 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302020db-0f2f-40f0-ac78-4c77dc29d24d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:24:54.209074 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302020db-0f2f-40f0-ac78-4c77dc29d24d admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:24:54.209348 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302020db-0f2f-40f0-ac78-4c77dc29d24d admin admin] PATH_INFO: `/v3/projects/792143568a794c458e1109cc41c6f947` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:24:54.211092 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302020db-0f2f-40f0-ac78-4c77dc29d24d admin admin] RBAC: Authorizing `identity:delete_project(project_id=792143568a794c458e1109cc41c6f947)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:24:54.212625 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302020db-0f2f-40f0-ac78-4c77dc29d24d admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:24:54.264538 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-302020db-0f2f-40f0-ac78-4c77dc29d24d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '792143568a794c458e1109cc41c6f947', 'request_id': 'req-302020db-0f2f-40f0-ac78-4c77dc29d24d'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:24:54.271427 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-302020db-0f2f-40f0-ac78-4c77dc29d24d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '792143568a794c458e1109cc41c6f947', 'request_id': 'req-302020db-0f2f-40f0-ac78-4c77dc29d24d'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:24:54.277557 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:24:54.279867 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-302020db-0f2f-40f0-ac78-4c77dc29d24d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '792143568a794c458e1109cc41c6f947', 'request_id': 'req-302020db-0f2f-40f0-ac78-4c77dc29d24d'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:24:54.296243 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-302020db-0f2f-40f0-ac78-4c77dc29d24d admin admin] The token cache is being invalidate because project 792143568a794c458e1109cc41c6f947 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80381) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 09:24:54.296562 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-302020db-0f2f-40f0-ac78-4c77dc29d24d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:24:54.297206 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-302020db-0f2f-40f0-ac78-4c77dc29d24d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:24:54.298774 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1257/2538] 10.222.0.26 () {58 vars in 1232 bytes} [Thu Jul 14 09:24:54 2022] DELETE /identity/v3/projects/792143568a794c458e1109cc41c6f947 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:24:54.304196 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313ae07a-1770-4ae9-bd06-b578ebbd5770 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:24:54.304594 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313ae07a-1770-4ae9-bd06-b578ebbd5770 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:24:54.305027 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313ae07a-1770-4ae9-bd06-b578ebbd5770 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:24:54.324988 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-313ae07a-1770-4ae9-bd06-b578ebbd5770 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:24:54.326122 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-313ae07a-1770-4ae9-bd06-b578ebbd5770 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:24:54.362759 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:24:54.391652 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1282/2539] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:24:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:24:54.400031 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db84796-1038-49ba-9c52-26a0483cacbe None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:24:54.400321 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db84796-1038-49ba-9c52-26a0483cacbe None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:24:54.400613 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db84796-1038-49ba-9c52-26a0483cacbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:24:54.419843 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-0db84796-1038-49ba-9c52-26a0483cacbe None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:24:54.421025 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-0db84796-1038-49ba-9c52-26a0483cacbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:24:54.429395 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1258/2540] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:24:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:24:54.434149 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313ae07a-1770-4ae9-bd06-b578ebbd5770 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:24:54.446048 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313ae07a-1770-4ae9-bd06-b578ebbd5770 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:24:54.451144 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b067180-2720-4733-809c-70e8b2e16d4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:24:54.451850 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b067180-2720-4733-809c-70e8b2e16d4d admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:24:54.452151 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b067180-2720-4733-809c-70e8b2e16d4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:24:54.452404 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b067180-2720-4733-809c-70e8b2e16d4d admin admin] PATH_INFO: `/v3/domains` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:24:54.453144 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b067180-2720-4733-809c-70e8b2e16d4d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:24:54.453471 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b067180-2720-4733-809c-70e8b2e16d4d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:24:54.455074 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b067180-2720-4733-809c-70e8b2e16d4d admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:24:54.467417 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1283/2541] 10.222.0.26 () {58 vars in 1152 bytes} [Thu Jul 14 09:24:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:24:54.471834 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db84796-1038-49ba-9c52-26a0483cacbe None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:24:54.481430 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db84796-1038-49ba-9c52-26a0483cacbe None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:24:54.486719 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14d3b76-9950-4809-b1b5-9c5f8efb0b40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:24:54.487457 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14d3b76-9950-4809-b1b5-9c5f8efb0b40 admin admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:24:54.487734 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14d3b76-9950-4809-b1b5-9c5f8efb0b40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:24:54.488010 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14d3b76-9950-4809-b1b5-9c5f8efb0b40 admin admin] PATH_INFO: `/v3/projects` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:24:54.488710 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14d3b76-9950-4809-b1b5-9c5f8efb0b40 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:24:54.490649 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14d3b76-9950-4809-b1b5-9c5f8efb0b40 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:24:54.518718 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1259/2542] 10.222.0.26 () {60 vars in 1152 bytes} [Thu Jul 14 09:24:54 2022] POST /identity/v3/projects => generated 365 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 09:24:54.523616 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b067180-2720-4733-809c-70e8b2e16d4d admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:24:54.534485 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b067180-2720-4733-809c-70e8b2e16d4d admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:24:54.596366 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c568ae42-25f3-4552-8b7d-cb6cea30a204 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:24:54.596965 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c568ae42-25f3-4552-8b7d-cb6cea30a204 admin admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:24:54.597263 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c568ae42-25f3-4552-8b7d-cb6cea30a204 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:24:54.597531 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c568ae42-25f3-4552-8b7d-cb6cea30a204 admin admin] PATH_INFO: `/v3/users` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:24:54.598238 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c568ae42-25f3-4552-8b7d-cb6cea30a204 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:24:54.600225 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c568ae42-25f3-4552-8b7d-cb6cea30a204 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:24:54.622550 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1284/2543] 10.222.0.26 () {60 vars in 1143 bytes} [Thu Jul 14 09:24:54 2022] POST /identity/v3/users => generated 283 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 09:24:54.626982 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14d3b76-9950-4809-b1b5-9c5f8efb0b40 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:24:54.639234 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14d3b76-9950-4809-b1b5-9c5f8efb0b40 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:24:54.644605 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3890d1af-adc0-453d-8d75-656f17c8a805 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:24:54.645600 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3890d1af-adc0-453d-8d75-656f17c8a805 admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:24:54.645887 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3890d1af-adc0-453d-8d75-656f17c8a805 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:24:54.646162 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3890d1af-adc0-453d-8d75-656f17c8a805 admin admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:24:54.646945 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3890d1af-adc0-453d-8d75-656f17c8a805 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:24:54.648661 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3890d1af-adc0-453d-8d75-656f17c8a805 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:24:54.659158 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1260/2544] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jul 14 09:24:54 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 09:24:54.663094 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c568ae42-25f3-4552-8b7d-cb6cea30a204 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:24:54.673539 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c568ae42-25f3-4552-8b7d-cb6cea30a204 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:24:54.678777 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6979329b-2dbb-4d97-9518-37a73391dc6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:24:54.679776 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6979329b-2dbb-4d97-9518-37a73391dc6d admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:24:54.680054 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6979329b-2dbb-4d97-9518-37a73391dc6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:24:54.680362 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6979329b-2dbb-4d97-9518-37a73391dc6d admin admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:24:54.681427 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6979329b-2dbb-4d97-9518-37a73391dc6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:24:54.684131 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6979329b-2dbb-4d97-9518-37a73391dc6d admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:24:54.699602 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1285/2545] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jul 14 09:24:54 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 09:24:54.704083 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3890d1af-adc0-453d-8d75-656f17c8a805 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:24:54.714527 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3890d1af-adc0-453d-8d75-656f17c8a805 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:24:54.720191 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5c22d7-3c8a-49f1-85df-3d4c25776e69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:24:54.721072 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5c22d7-3c8a-49f1-85df-3d4c25776e69 admin admin] REQUEST_METHOD: `PUT` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:24:54.721384 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5c22d7-3c8a-49f1-85df-3d4c25776e69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:24:54.721651 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5c22d7-3c8a-49f1-85df-3d4c25776e69 admin admin] PATH_INFO: `/v3/projects/e35a5af1c07e41ce96c3cc5ab8895129/users/66ddbcaf27d54ea0ae314192d2088c8d/roles/d1691322ce6a4edab7ce722edf0f2988` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:24:54.736627 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5c22d7-3c8a-49f1-85df-3d4c25776e69 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e35a5af1c07e41ce96c3cc5ab8895129, user_id=66ddbcaf27d54ea0ae314192d2088c8d, role_id=d1691322ce6a4edab7ce722edf0f2988)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:24:54.739309 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5c22d7-3c8a-49f1-85df-3d4c25776e69 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:24:54.755950 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1261/2546] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jul 14 09:24:54 2022] PUT /identity/v3/projects/e35a5af1c07e41ce96c3cc5ab8895129/users/66ddbcaf27d54ea0ae314192d2088c8d/roles/d1691322ce6a4edab7ce722edf0f2988 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:24:54.767639 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5a4e0f-d749-454e-96af-1545e455db32 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:24:54.767937 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5a4e0f-d749-454e-96af-1545e455db32 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:24:54.768215 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5a4e0f-d749-454e-96af-1545e455db32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:24:54.925156 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-1a5a4e0f-d749-454e-96af-1545e455db32 None None] MFA Rules not processed for user `66ddbcaf27d54ea0ae314192d2088c8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:24:54.926372 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-1a5a4e0f-d749-454e-96af-1545e455db32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:24:54.996605 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1286/2547] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:24:54 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:24:55.001134 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5c22d7-3c8a-49f1-85df-3d4c25776e69 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:24:55.065328 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5c22d7-3c8a-49f1-85df-3d4c25776e69 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:24:55.070377 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fde2a6b-611f-41d3-90ef-70156cc6ab1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:24:55.071013 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fde2a6b-611f-41d3-90ef-70156cc6ab1d admin admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:24:55.071320 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fde2a6b-611f-41d3-90ef-70156cc6ab1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:24:55.071626 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fde2a6b-611f-41d3-90ef-70156cc6ab1d admin admin] PATH_INFO: `/v3/projects` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:24:55.072315 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fde2a6b-611f-41d3-90ef-70156cc6ab1d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:24:55.074252 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fde2a6b-611f-41d3-90ef-70156cc6ab1d admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:24:55.103003 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1262/2548] 10.222.0.26 () {60 vars in 1152 bytes} [Thu Jul 14 09:24:55 2022] POST /identity/v3/projects => generated 365 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 09:24:55.107167 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5a4e0f-d749-454e-96af-1545e455db32 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:24:55.117915 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5a4e0f-d749-454e-96af-1545e455db32 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:24:55.150019 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:24:55.178924 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9c434e-7df1-4618-a716-a7c8cf3bb0d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:24:55.179658 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9c434e-7df1-4618-a716-a7c8cf3bb0d3 admin admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:24:55.179940 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9c434e-7df1-4618-a716-a7c8cf3bb0d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:24:55.180211 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9c434e-7df1-4618-a716-a7c8cf3bb0d3 admin admin] PATH_INFO: `/v3/users` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:24:55.180962 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9c434e-7df1-4618-a716-a7c8cf3bb0d3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:24:55.183207 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9c434e-7df1-4618-a716-a7c8cf3bb0d3 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:24:55.208921 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1287/2549] 10.222.0.26 () {60 vars in 1143 bytes} [Thu Jul 14 09:24:55 2022] POST /identity/v3/users => generated 289 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 09:24:55.213300 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fde2a6b-611f-41d3-90ef-70156cc6ab1d admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:24:55.225936 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fde2a6b-611f-41d3-90ef-70156cc6ab1d admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:24:55.231935 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c704bb2-7d3f-47a1-9680-c951bbb388c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:24:55.232636 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c704bb2-7d3f-47a1-9680-c951bbb388c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:24:55.232921 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c704bb2-7d3f-47a1-9680-c951bbb388c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:24:55.233197 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c704bb2-7d3f-47a1-9680-c951bbb388c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:24:55.233941 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c704bb2-7d3f-47a1-9680-c951bbb388c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:24:55.235981 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c704bb2-7d3f-47a1-9680-c951bbb388c3 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:24:55.247387 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1263/2550] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jul 14 09:24:55 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 09:24:55.251896 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9c434e-7df1-4618-a716-a7c8cf3bb0d3 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:24:55.262538 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9c434e-7df1-4618-a716-a7c8cf3bb0d3 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:24:55.267752 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07a6107-18d7-4054-ac8d-3507c3cc663a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:24:55.268660 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07a6107-18d7-4054-ac8d-3507c3cc663a admin admin] REQUEST_METHOD: `PUT` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:24:55.268935 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07a6107-18d7-4054-ac8d-3507c3cc663a admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:24:55.269206 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07a6107-18d7-4054-ac8d-3507c3cc663a admin admin] PATH_INFO: `/v3/projects/7612b400d7364aa58d9897c7614cf35a/users/b3bcf29cf8774866992e64be0227eae7/roles/7b549d58d1724395b343609d895c80d0` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:24:55.285127 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07a6107-18d7-4054-ac8d-3507c3cc663a admin admin] RBAC: Authorizing `identity:create_grant(project_id=7612b400d7364aa58d9897c7614cf35a, user_id=b3bcf29cf8774866992e64be0227eae7, role_id=7b549d58d1724395b343609d895c80d0)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:24:55.288158 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07a6107-18d7-4054-ac8d-3507c3cc663a admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:24:55.302911 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1288/2551] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jul 14 09:24:55 2022] PUT /identity/v3/projects/7612b400d7364aa58d9897c7614cf35a/users/b3bcf29cf8774866992e64be0227eae7/roles/7b549d58d1724395b343609d895c80d0 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:24:55.307131 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c704bb2-7d3f-47a1-9680-c951bbb388c3 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:24:55.317554 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c704bb2-7d3f-47a1-9680-c951bbb388c3 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:24:55.377523 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20a467b-4d95-4f68-9197-66b6671c88aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:24:55.378237 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20a467b-4d95-4f68-9197-66b6671c88aa admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:24:55.378778 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20a467b-4d95-4f68-9197-66b6671c88aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:24:55.378778 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20a467b-4d95-4f68-9197-66b6671c88aa admin admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:24:55.379795 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20a467b-4d95-4f68-9197-66b6671c88aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:24:55.381460 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20a467b-4d95-4f68-9197-66b6671c88aa admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:24:55.382821 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:24:55.398060 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1264/2552] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jul 14 09:24:55 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 09:24:55.402368 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07a6107-18d7-4054-ac8d-3507c3cc663a admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:24:55.412937 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07a6107-18d7-4054-ac8d-3507c3cc663a admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:24:55.418025 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc4d68c-5b26-4f3b-a97a-6318eda9bab7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:24:55.418977 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc4d68c-5b26-4f3b-a97a-6318eda9bab7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:24:55.419272 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc4d68c-5b26-4f3b-a97a-6318eda9bab7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:24:55.419553 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc4d68c-5b26-4f3b-a97a-6318eda9bab7 admin admin] PATH_INFO: `/v3/projects/7612b400d7364aa58d9897c7614cf35a/users/b3bcf29cf8774866992e64be0227eae7/roles/7b549d58d1724395b343609d895c80d0` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:24:55.422066 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc4d68c-5b26-4f3b-a97a-6318eda9bab7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7612b400d7364aa58d9897c7614cf35a, user_id=b3bcf29cf8774866992e64be0227eae7, role_id=7b549d58d1724395b343609d895c80d0)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:24:55.425609 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc4d68c-5b26-4f3b-a97a-6318eda9bab7 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:24:55.439359 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1289/2553] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jul 14 09:24:55 2022] PUT /identity/v3/projects/7612b400d7364aa58d9897c7614cf35a/users/b3bcf29cf8774866992e64be0227eae7/roles/7b549d58d1724395b343609d895c80d0 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 09:24:55.450334 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17062c9d-c413-4b13-b7f6-57cf130adcce None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:24:55.450587 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17062c9d-c413-4b13-b7f6-57cf130adcce None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:24:55.450883 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17062c9d-c413-4b13-b7f6-57cf130adcce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:24:55.479013 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-17062c9d-c413-4b13-b7f6-57cf130adcce None None] MFA Rules not processed for user `b3bcf29cf8774866992e64be0227eae7`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:24:55.480551 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-17062c9d-c413-4b13-b7f6-57cf130adcce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:24:55.563627 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1265/2554] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:24:55 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:24:55.577628 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc4d68c-5b26-4f3b-a97a-6318eda9bab7 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:24:55.587832 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-8bc4d68c-5b26-4f3b-a97a-6318eda9bab7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:24:55.644107 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc4d68c-5b26-4f3b-a97a-6318eda9bab7 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:24:55.648046 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc126d7-2ff6-4753-b319-0786fb89cae1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e76f1952a5b146808844b9f2b7395035', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:24:55.648697 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc126d7-2ff6-4753-b319-0786fb89cae1 service cinder] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:24:55.648980 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc126d7-2ff6-4753-b319-0786fb89cae1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:24:55.649258 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc126d7-2ff6-4753-b319-0786fb89cae1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:24:55.661998 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc126d7-2ff6-4753-b319-0786fb89cae1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:24:55.664012 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc126d7-2ff6-4753-b319-0786fb89cae1 service cinder] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:24:55.678263 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1290/2555] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:24:55 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:24:56.121738 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17062c9d-c413-4b13-b7f6-57cf130adcce None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:24:56.183822 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17062c9d-c413-4b13-b7f6-57cf130adcce None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:24:56.188203 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029df9b8-d230-44f8-b6ac-43e04be33b27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:24:56.189104 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029df9b8-d230-44f8-b6ac-43e04be33b27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:24:56.189385 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029df9b8-d230-44f8-b6ac-43e04be33b27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:24:56.189659 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029df9b8-d230-44f8-b6ac-43e04be33b27 admin admin] PATH_INFO: `/v3/users/66ddbcaf27d54ea0ae314192d2088c8d` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:24:56.190833 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029df9b8-d230-44f8-b6ac-43e04be33b27 admin admin] RBAC: Authorizing `identity:delete_user(user_id=66ddbcaf27d54ea0ae314192d2088c8d)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:24:56.192798 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029df9b8-d230-44f8-b6ac-43e04be33b27 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:24:56.254314 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-029df9b8-d230-44f8-b6ac-43e04be33b27 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '66ddbcaf27d54ea0ae314192d2088c8d'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:24:56.265143 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-029df9b8-d230-44f8-b6ac-43e04be33b27 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '66ddbcaf27d54ea0ae314192d2088c8d'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:24:56.265853 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-029df9b8-d230-44f8-b6ac-43e04be33b27 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '66ddbcaf27d54ea0ae314192d2088c8d'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:24:56.266799 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-029df9b8-d230-44f8-b6ac-43e04be33b27 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '66ddbcaf27d54ea0ae314192d2088c8d'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:24:56.277759 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-029df9b8-d230-44f8-b6ac-43e04be33b27 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '66ddbcaf27d54ea0ae314192d2088c8d'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:24:56.292012 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:24:56.296873 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1266/2556] 10.222.0.26 () {58 vars in 1223 bytes} [Thu Jul 14 09:24:56 2022] DELETE /identity/v3/users/66ddbcaf27d54ea0ae314192d2088c8d => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:24:56.301814 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcb7a78-46e4-4db3-8e15-09f06f197598 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:24:56.302102 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcb7a78-46e4-4db3-8e15-09f06f197598 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:24:56.302379 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcb7a78-46e4-4db3-8e15-09f06f197598 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:24:56.320767 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-7fcb7a78-46e4-4db3-8e15-09f06f197598 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:24:56.321972 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-7fcb7a78-46e4-4db3-8e15-09f06f197598 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:24:56.387514 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1291/2557] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:24:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:24:56.397796 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029df9b8-d230-44f8-b6ac-43e04be33b27 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:24:56.399499 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-029df9b8-d230-44f8-b6ac-43e04be33b27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:24:56.462336 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029df9b8-d230-44f8-b6ac-43e04be33b27 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:24:56.466255 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8b5c09-d7f0-4e10-893e-0eaecef5b0bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '952911c492444ffea4b211a24b564117', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:24:56.466891 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8b5c09-d7f0-4e10-893e-0eaecef5b0bc service neutron] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:24:56.467216 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8b5c09-d7f0-4e10-893e-0eaecef5b0bc service neutron] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:24:56.467489 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8b5c09-d7f0-4e10-893e-0eaecef5b0bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:24:56.477387 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8b5c09-d7f0-4e10-893e-0eaecef5b0bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:24:56.478819 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8b5c09-d7f0-4e10-893e-0eaecef5b0bc service neutron] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:24:56.488590 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1267/2558] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:24:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:24:56.758118 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcb7a78-46e4-4db3-8e15-09f06f197598 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:24:56.759593 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-7fcb7a78-46e4-4db3-8e15-09f06f197598 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:24:56.771989 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcb7a78-46e4-4db3-8e15-09f06f197598 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:24:56.777719 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a232a8-7e30-4ce0-8d21-8131aa5576d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:24:56.778634 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a232a8-7e30-4ce0-8d21-8131aa5576d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:24:56.778942 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a232a8-7e30-4ce0-8d21-8131aa5576d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:24:56.779230 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a232a8-7e30-4ce0-8d21-8131aa5576d4 admin admin] PATH_INFO: `/v3/projects/e35a5af1c07e41ce96c3cc5ab8895129` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:24:56.780656 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a232a8-7e30-4ce0-8d21-8131aa5576d4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e35a5af1c07e41ce96c3cc5ab8895129)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:24:56.782596 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a232a8-7e30-4ce0-8d21-8131aa5576d4 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:24:56.809082 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:24:56.834659 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-c2a232a8-7e30-4ce0-8d21-8131aa5576d4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e35a5af1c07e41ce96c3cc5ab8895129', 'request_id': 'req-c2a232a8-7e30-4ce0-8d21-8131aa5576d4'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:24:56.840793 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-c2a232a8-7e30-4ce0-8d21-8131aa5576d4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e35a5af1c07e41ce96c3cc5ab8895129', 'request_id': 'req-c2a232a8-7e30-4ce0-8d21-8131aa5576d4'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:24:56.846430 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-c2a232a8-7e30-4ce0-8d21-8131aa5576d4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e35a5af1c07e41ce96c3cc5ab8895129', 'request_id': 'req-c2a232a8-7e30-4ce0-8d21-8131aa5576d4'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:24:56.859847 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-c2a232a8-7e30-4ce0-8d21-8131aa5576d4 admin admin] The token cache is being invalidate because project e35a5af1c07e41ce96c3cc5ab8895129 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80382) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 09:24:56.860168 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-c2a232a8-7e30-4ce0-8d21-8131aa5576d4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:24:56.860803 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-c2a232a8-7e30-4ce0-8d21-8131aa5576d4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:24:56.862383 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1292/2559] 10.222.0.26 () {58 vars in 1232 bytes} [Thu Jul 14 09:24:56 2022] DELETE /identity/v3/projects/e35a5af1c07e41ce96c3cc5ab8895129 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:24:56.866446 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8b5c09-d7f0-4e10-893e-0eaecef5b0bc service neutron] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:24:56.876310 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8b5c09-d7f0-4e10-893e-0eaecef5b0bc service neutron] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:24:56.937480 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f11a524-ad32-4128-bfe5-64904ba7be0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:24:56.938338 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f11a524-ad32-4128-bfe5-64904ba7be0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:24:56.938616 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f11a524-ad32-4128-bfe5-64904ba7be0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:24:56.938927 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f11a524-ad32-4128-bfe5-64904ba7be0e admin admin] PATH_INFO: `/v3/users/b3bcf29cf8774866992e64be0227eae7` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:24:56.940229 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f11a524-ad32-4128-bfe5-64904ba7be0e admin admin] RBAC: Authorizing `identity:delete_user(user_id=b3bcf29cf8774866992e64be0227eae7)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:24:56.942130 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f11a524-ad32-4128-bfe5-64904ba7be0e admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:24:57.003247 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-6f11a524-ad32-4128-bfe5-64904ba7be0e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b3bcf29cf8774866992e64be0227eae7'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:24:57.011985 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-6f11a524-ad32-4128-bfe5-64904ba7be0e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b3bcf29cf8774866992e64be0227eae7'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:24:57.012994 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-6f11a524-ad32-4128-bfe5-64904ba7be0e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b3bcf29cf8774866992e64be0227eae7'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:24:57.013653 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-6f11a524-ad32-4128-bfe5-64904ba7be0e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b3bcf29cf8774866992e64be0227eae7'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:24:57.021264 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-6f11a524-ad32-4128-bfe5-64904ba7be0e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b3bcf29cf8774866992e64be0227eae7'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:24:57.039642 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1268/2560] 10.222.0.26 () {58 vars in 1223 bytes} [Thu Jul 14 09:24:56 2022] DELETE /identity/v3/users/b3bcf29cf8774866992e64be0227eae7 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 09:24:57.324120 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a232a8-7e30-4ce0-8d21-8131aa5576d4 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:24:57.325655 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-c2a232a8-7e30-4ce0-8d21-8131aa5576d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:24:57.341627 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a232a8-7e30-4ce0-8d21-8131aa5576d4 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:24:57.397959 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a4c4f5-855f-4a43-baeb-84b32852688e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:24:57.398859 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a4c4f5-855f-4a43-baeb-84b32852688e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:24:57.399132 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a4c4f5-855f-4a43-baeb-84b32852688e admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:24:57.399432 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a4c4f5-855f-4a43-baeb-84b32852688e admin admin] PATH_INFO: `/v3/projects/7612b400d7364aa58d9897c7614cf35a` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:24:57.400609 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a4c4f5-855f-4a43-baeb-84b32852688e admin admin] RBAC: Authorizing `identity:delete_project(project_id=7612b400d7364aa58d9897c7614cf35a)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:24:57.402524 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a4c4f5-855f-4a43-baeb-84b32852688e admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:24:57.452083 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-94a4c4f5-855f-4a43-baeb-84b32852688e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7612b400d7364aa58d9897c7614cf35a', 'request_id': 'req-94a4c4f5-855f-4a43-baeb-84b32852688e'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:24:57.458249 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-94a4c4f5-855f-4a43-baeb-84b32852688e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7612b400d7364aa58d9897c7614cf35a', 'request_id': 'req-94a4c4f5-855f-4a43-baeb-84b32852688e'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:24:57.463574 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-94a4c4f5-855f-4a43-baeb-84b32852688e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7612b400d7364aa58d9897c7614cf35a', 'request_id': 'req-94a4c4f5-855f-4a43-baeb-84b32852688e'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:24:57.477756 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-94a4c4f5-855f-4a43-baeb-84b32852688e admin admin] The token cache is being invalidate because project 7612b400d7364aa58d9897c7614cf35a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80382) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 09:24:57.478032 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-94a4c4f5-855f-4a43-baeb-84b32852688e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:24:57.478675 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-94a4c4f5-855f-4a43-baeb-84b32852688e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:24:57.480335 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1293/2561] 10.222.0.26 () {58 vars in 1232 bytes} [Thu Jul 14 09:24:57 2022] DELETE /identity/v3/projects/7612b400d7364aa58d9897c7614cf35a => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:24:57.485235 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f81ee0-0fd3-4dff-a849-bc224f1d0d86 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:24:57.485549 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f81ee0-0fd3-4dff-a849-bc224f1d0d86 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:24:57.485825 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f81ee0-0fd3-4dff-a849-bc224f1d0d86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:24:57.503638 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-66f81ee0-0fd3-4dff-a849-bc224f1d0d86 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:24:57.504924 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-66f81ee0-0fd3-4dff-a849-bc224f1d0d86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:24:57.529351 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:24:57.575016 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1269/2562] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:24:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:24:57.583038 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad38cadd-8b3a-4893-9f30-1a7bf8ff2eb8 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:24:57.583358 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad38cadd-8b3a-4893-9f30-1a7bf8ff2eb8 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:24:57.583640 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad38cadd-8b3a-4893-9f30-1a7bf8ff2eb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:24:57.603148 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-ad38cadd-8b3a-4893-9f30-1a7bf8ff2eb8 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:24:57.604354 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-ad38cadd-8b3a-4893-9f30-1a7bf8ff2eb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:24:57.612795 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1294/2563] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:24:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:24:57.616897 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f81ee0-0fd3-4dff-a849-bc224f1d0d86 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:24:57.627382 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f81ee0-0fd3-4dff-a849-bc224f1d0d86 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:24:57.633236 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d79b17-11ca-4751-991d-9f9236d772ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:24:57.634027 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d79b17-11ca-4751-991d-9f9236d772ad admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:24:57.634316 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d79b17-11ca-4751-991d-9f9236d772ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:24:57.634595 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d79b17-11ca-4751-991d-9f9236d772ad admin admin] PATH_INFO: `/v3/domains` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:24:57.635347 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d79b17-11ca-4751-991d-9f9236d772ad admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:24:57.635652 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d79b17-11ca-4751-991d-9f9236d772ad admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:24:57.637320 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d79b17-11ca-4751-991d-9f9236d772ad admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:24:57.649913 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1270/2564] 10.222.0.26 () {58 vars in 1152 bytes} [Thu Jul 14 09:24:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:24:57.654208 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad38cadd-8b3a-4893-9f30-1a7bf8ff2eb8 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:24:57.663776 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad38cadd-8b3a-4893-9f30-1a7bf8ff2eb8 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:24:57.669423 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023057df-4e4e-48c3-bf9a-41f2e183a141 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:24:57.670086 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023057df-4e4e-48c3-bf9a-41f2e183a141 admin admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:24:57.670365 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023057df-4e4e-48c3-bf9a-41f2e183a141 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:24:57.670703 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023057df-4e4e-48c3-bf9a-41f2e183a141 admin admin] PATH_INFO: `/v3/projects` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:24:57.671428 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023057df-4e4e-48c3-bf9a-41f2e183a141 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:24:57.673321 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023057df-4e4e-48c3-bf9a-41f2e183a141 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:24:57.705429 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1295/2565] 10.222.0.26 () {60 vars in 1152 bytes} [Thu Jul 14 09:24:57 2022] POST /identity/v3/projects => generated 399 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 09:24:57.709547 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d79b17-11ca-4751-991d-9f9236d772ad admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:24:57.720962 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d79b17-11ca-4751-991d-9f9236d772ad admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:24:57.791164 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4000d8-4298-4346-988d-d55824282432 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:24:57.791782 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4000d8-4298-4346-988d-d55824282432 admin admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:24:57.792076 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4000d8-4298-4346-988d-d55824282432 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:24:57.792361 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4000d8-4298-4346-988d-d55824282432 admin admin] PATH_INFO: `/v3/users` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:24:57.793076 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4000d8-4298-4346-988d-d55824282432 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:24:57.795394 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4000d8-4298-4346-988d-d55824282432 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:24:57.817669 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1271/2566] 10.222.0.26 () {60 vars in 1143 bytes} [Thu Jul 14 09:24:57 2022] POST /identity/v3/users => generated 300 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 09:24:57.824662 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023057df-4e4e-48c3-bf9a-41f2e183a141 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:24:57.834496 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023057df-4e4e-48c3-bf9a-41f2e183a141 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:24:57.839720 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53280776-eeb7-4d91-8f20-42b8b88bd82d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:24:57.840514 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53280776-eeb7-4d91-8f20-42b8b88bd82d admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:24:57.840837 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53280776-eeb7-4d91-8f20-42b8b88bd82d admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:24:57.841241 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53280776-eeb7-4d91-8f20-42b8b88bd82d admin admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:24:57.842005 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53280776-eeb7-4d91-8f20-42b8b88bd82d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:24:57.843872 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53280776-eeb7-4d91-8f20-42b8b88bd82d admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:24:57.854650 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1296/2567] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jul 14 09:24:57 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 09:24:57.858671 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4000d8-4298-4346-988d-d55824282432 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:24:57.869669 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4000d8-4298-4346-988d-d55824282432 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:24:57.875221 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748c5a44-d62b-45f6-ad4e-f886c8d35790 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:24:57.875955 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748c5a44-d62b-45f6-ad4e-f886c8d35790 admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:24:57.876242 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748c5a44-d62b-45f6-ad4e-f886c8d35790 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:24:57.876522 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748c5a44-d62b-45f6-ad4e-f886c8d35790 admin admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:24:57.877259 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748c5a44-d62b-45f6-ad4e-f886c8d35790 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:24:57.878872 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748c5a44-d62b-45f6-ad4e-f886c8d35790 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:24:57.889652 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1272/2568] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jul 14 09:24:57 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 09:24:57.894065 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53280776-eeb7-4d91-8f20-42b8b88bd82d admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:24:57.899678 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=84,keystone:DELETE=13,keystone:INSERT=3 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:24:57.908093 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53280776-eeb7-4d91-8f20-42b8b88bd82d admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:24:57.915746 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996ccea9-14dc-455d-bed5-282ef3a03bf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:24:57.917079 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996ccea9-14dc-455d-bed5-282ef3a03bf9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:24:57.917509 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996ccea9-14dc-455d-bed5-282ef3a03bf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:24:57.917914 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996ccea9-14dc-455d-bed5-282ef3a03bf9 admin admin] PATH_INFO: `/v3/projects/b524b2d28a5441e89749b96ec0c429fe/users/bfd32e8c76ba46ec8c005bcb054486ea/roles/d1691322ce6a4edab7ce722edf0f2988` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:24:57.937564 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996ccea9-14dc-455d-bed5-282ef3a03bf9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b524b2d28a5441e89749b96ec0c429fe, user_id=bfd32e8c76ba46ec8c005bcb054486ea, role_id=d1691322ce6a4edab7ce722edf0f2988)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:24:57.940566 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996ccea9-14dc-455d-bed5-282ef3a03bf9 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:24:57.954186 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1297/2569] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jul 14 09:24:57 2022] PUT /identity/v3/projects/b524b2d28a5441e89749b96ec0c429fe/users/bfd32e8c76ba46ec8c005bcb054486ea/roles/d1691322ce6a4edab7ce722edf0f2988 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:24:57.965586 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7aeff0-79a5-4cd1-875e-4baf71884625 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:24:57.965875 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7aeff0-79a5-4cd1-875e-4baf71884625 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:24:57.966152 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7aeff0-79a5-4cd1-875e-4baf71884625 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:24:57.995262 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-4c7aeff0-79a5-4cd1-875e-4baf71884625 None None] MFA Rules not processed for user `bfd32e8c76ba46ec8c005bcb054486ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:24:57.996602 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-4c7aeff0-79a5-4cd1-875e-4baf71884625 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:24:58.065764 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1273/2570] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:24:57 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:24:58.070421 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996ccea9-14dc-455d-bed5-282ef3a03bf9 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:24:58.136479 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996ccea9-14dc-455d-bed5-282ef3a03bf9 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:24:58.141579 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26a36b5-e32c-465e-ac15-ed33147c552e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:24:58.142239 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26a36b5-e32c-465e-ac15-ed33147c552e admin admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:24:58.142616 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26a36b5-e32c-465e-ac15-ed33147c552e admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:24:58.142787 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26a36b5-e32c-465e-ac15-ed33147c552e admin admin] PATH_INFO: `/v3/projects` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:24:58.143507 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26a36b5-e32c-465e-ac15-ed33147c552e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:24:58.145983 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26a36b5-e32c-465e-ac15-ed33147c552e admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:24:58.175766 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1298/2571] 10.222.0.26 () {60 vars in 1152 bytes} [Thu Jul 14 09:24:58 2022] POST /identity/v3/projects => generated 401 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 09:24:58.176845 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7aeff0-79a5-4cd1-875e-4baf71884625 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:24:58.187682 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7aeff0-79a5-4cd1-875e-4baf71884625 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:24:58.200225 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:24:58.248519 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0476adc5-b57f-41d0-8008-6b1e0a6a00ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:24:58.249296 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0476adc5-b57f-41d0-8008-6b1e0a6a00ef admin admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:24:58.249574 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0476adc5-b57f-41d0-8008-6b1e0a6a00ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:24:58.249843 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0476adc5-b57f-41d0-8008-6b1e0a6a00ef admin admin] PATH_INFO: `/v3/users` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:24:58.250541 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0476adc5-b57f-41d0-8008-6b1e0a6a00ef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:24:58.252500 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0476adc5-b57f-41d0-8008-6b1e0a6a00ef admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:24:58.273530 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1274/2572] 10.222.0.26 () {60 vars in 1143 bytes} [Thu Jul 14 09:24:58 2022] POST /identity/v3/users => generated 307 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 09:24:58.277578 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26a36b5-e32c-465e-ac15-ed33147c552e admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:24:58.288782 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26a36b5-e32c-465e-ac15-ed33147c552e admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:24:58.294465 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97f1fa2-f6dc-4a1a-b5f4-8e29505eb8d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:24:58.295128 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97f1fa2-f6dc-4a1a-b5f4-8e29505eb8d7 admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:24:58.295434 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97f1fa2-f6dc-4a1a-b5f4-8e29505eb8d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:24:58.295758 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97f1fa2-f6dc-4a1a-b5f4-8e29505eb8d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:24:58.296427 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97f1fa2-f6dc-4a1a-b5f4-8e29505eb8d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:24:58.298059 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97f1fa2-f6dc-4a1a-b5f4-8e29505eb8d7 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:24:58.308287 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1299/2573] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jul 14 09:24:58 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 09:24:58.312439 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0476adc5-b57f-41d0-8008-6b1e0a6a00ef admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:24:58.322297 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0476adc5-b57f-41d0-8008-6b1e0a6a00ef admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:24:58.326999 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4b6c80-e2e6-4648-83ca-4e1c8930b9d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:24:58.328013 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4b6c80-e2e6-4648-83ca-4e1c8930b9d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:24:58.328347 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4b6c80-e2e6-4648-83ca-4e1c8930b9d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:24:58.328636 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4b6c80-e2e6-4648-83ca-4e1c8930b9d7 admin admin] PATH_INFO: `/v3/projects/d6f178168de74339bf5a482c28ad2954/users/9b34aeda4eda47e49b359c8658e89e58/roles/7b549d58d1724395b343609d895c80d0` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:24:58.343387 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4b6c80-e2e6-4648-83ca-4e1c8930b9d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d6f178168de74339bf5a482c28ad2954, user_id=9b34aeda4eda47e49b359c8658e89e58, role_id=7b549d58d1724395b343609d895c80d0)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:24:58.346016 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4b6c80-e2e6-4648-83ca-4e1c8930b9d7 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:24:58.360035 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1275/2574] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jul 14 09:24:58 2022] PUT /identity/v3/projects/d6f178168de74339bf5a482c28ad2954/users/9b34aeda4eda47e49b359c8658e89e58/roles/7b549d58d1724395b343609d895c80d0 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 09:24:58.364138 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97f1fa2-f6dc-4a1a-b5f4-8e29505eb8d7 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:24:58.376398 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97f1fa2-f6dc-4a1a-b5f4-8e29505eb8d7 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:24:58.436546 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62316b5c-1fd3-4372-99ed-7c5225a2b6dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:24:58.437219 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62316b5c-1fd3-4372-99ed-7c5225a2b6dd admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:24:58.437503 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62316b5c-1fd3-4372-99ed-7c5225a2b6dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:24:58.437778 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62316b5c-1fd3-4372-99ed-7c5225a2b6dd admin admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:24:58.438512 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62316b5c-1fd3-4372-99ed-7c5225a2b6dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:24:58.440278 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62316b5c-1fd3-4372-99ed-7c5225a2b6dd admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:24:58.451111 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1300/2575] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jul 14 09:24:58 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 09:24:58.455338 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4b6c80-e2e6-4648-83ca-4e1c8930b9d7 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:24:58.465841 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4b6c80-e2e6-4648-83ca-4e1c8930b9d7 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:24:58.470835 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aad0321-a709-46f0-a78f-b6bcfeb644c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:24:58.471737 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aad0321-a709-46f0-a78f-b6bcfeb644c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:24:58.472014 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aad0321-a709-46f0-a78f-b6bcfeb644c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:24:58.472289 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aad0321-a709-46f0-a78f-b6bcfeb644c7 admin admin] PATH_INFO: `/v3/projects/d6f178168de74339bf5a482c28ad2954/users/9b34aeda4eda47e49b359c8658e89e58/roles/7b549d58d1724395b343609d895c80d0` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:24:58.474264 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aad0321-a709-46f0-a78f-b6bcfeb644c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d6f178168de74339bf5a482c28ad2954, user_id=9b34aeda4eda47e49b359c8658e89e58, role_id=7b549d58d1724395b343609d895c80d0)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:24:58.476926 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aad0321-a709-46f0-a78f-b6bcfeb644c7 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:24:58.487229 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1276/2576] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jul 14 09:24:58 2022] PUT /identity/v3/projects/d6f178168de74339bf5a482c28ad2954/users/9b34aeda4eda47e49b359c8658e89e58/roles/7b549d58d1724395b343609d895c80d0 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:24:58.498143 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66bbf7e-6e2b-4902-8dd4-666b4431c40a None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:24:58.498438 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66bbf7e-6e2b-4902-8dd4-666b4431c40a None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:24:58.498788 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66bbf7e-6e2b-4902-8dd4-666b4431c40a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:24:58.539538 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-a66bbf7e-6e2b-4902-8dd4-666b4431c40a None None] MFA Rules not processed for user `9b34aeda4eda47e49b359c8658e89e58`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:24:58.541478 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-a66bbf7e-6e2b-4902-8dd4-666b4431c40a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:24:58.586511 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:24:58.618492 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1301/2577] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:24:58 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:24:58.629910 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aad0321-a709-46f0-a78f-b6bcfeb644c7 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:24:58.632074 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-2aad0321-a709-46f0-a78f-b6bcfeb644c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:24:58.689206 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aad0321-a709-46f0-a78f-b6bcfeb644c7 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:24:58.693624 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bdc867-6563-4220-a965-946704a68fd5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e76f1952a5b146808844b9f2b7395035', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:24:58.694258 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bdc867-6563-4220-a965-946704a68fd5 service cinder] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:24:58.694554 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bdc867-6563-4220-a965-946704a68fd5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:24:58.694822 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bdc867-6563-4220-a965-946704a68fd5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:24:58.704745 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bdc867-6563-4220-a965-946704a68fd5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:24:58.706172 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bdc867-6563-4220-a965-946704a68fd5 service cinder] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:24:58.715863 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1277/2578] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:24:58 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:24:58.789858 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66bbf7e-6e2b-4902-8dd4-666b4431c40a None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:24:58.799379 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66bbf7e-6e2b-4902-8dd4-666b4431c40a None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:24:58.803027 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a274cc-3056-48f6-a6e5-db86b2f25319 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e76f1952a5b146808844b9f2b7395035', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:24:58.803725 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a274cc-3056-48f6-a6e5-db86b2f25319 service cinder] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:24:58.804001 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a274cc-3056-48f6-a6e5-db86b2f25319 service cinder] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:24:58.804266 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a274cc-3056-48f6-a6e5-db86b2f25319 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:24:58.862571 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a274cc-3056-48f6-a6e5-db86b2f25319 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:24:58.864043 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a274cc-3056-48f6-a6e5-db86b2f25319 service cinder] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:24:58.873202 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1302/2579] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:24:58 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:25:02.537196 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5748caa-4354-4793-b538-3e8fda0661fd None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:02.537750 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5748caa-4354-4793-b538-3e8fda0661fd None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:02.537750 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5748caa-4354-4793-b538-3e8fda0661fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:02.558782 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-a5748caa-4354-4793-b538-3e8fda0661fd None None] MFA Rules not processed for user `42662fb9bae5403a8a6f698358f5c20c`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:25:02.560103 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-a5748caa-4354-4793-b538-3e8fda0661fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:25:02.633153 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1278/2580] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:25:02 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:25:04.008246 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a274cc-3056-48f6-a6e5-db86b2f25319 service cinder] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:25:04.074142 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a274cc-3056-48f6-a6e5-db86b2f25319 service cinder] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:25:04.078856 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad420044-0875-49ef-9dab-90a6e008a262 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:25:04.079806 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad420044-0875-49ef-9dab-90a6e008a262 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:04.080092 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad420044-0875-49ef-9dab-90a6e008a262 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:04.080367 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad420044-0875-49ef-9dab-90a6e008a262 admin admin] PATH_INFO: `/v3/users/bfd32e8c76ba46ec8c005bcb054486ea` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:04.081562 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad420044-0875-49ef-9dab-90a6e008a262 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bfd32e8c76ba46ec8c005bcb054486ea)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:25:04.083531 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad420044-0875-49ef-9dab-90a6e008a262 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:25:04.141113 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-ad420044-0875-49ef-9dab-90a6e008a262 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bfd32e8c76ba46ec8c005bcb054486ea'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:25:04.141944 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-ad420044-0875-49ef-9dab-90a6e008a262 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bfd32e8c76ba46ec8c005bcb054486ea'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:25:04.156333 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-ad420044-0875-49ef-9dab-90a6e008a262 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bfd32e8c76ba46ec8c005bcb054486ea'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:25:04.164804 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-ad420044-0875-49ef-9dab-90a6e008a262 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bfd32e8c76ba46ec8c005bcb054486ea'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:25:04.165684 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-ad420044-0875-49ef-9dab-90a6e008a262 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bfd32e8c76ba46ec8c005bcb054486ea'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:25:04.169455 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:25:04.186981 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1303/2581] 10.222.0.26 () {58 vars in 1223 bytes} [Thu Jul 14 09:25:04 2022] DELETE /identity/v3/users/bfd32e8c76ba46ec8c005bcb054486ea => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:25:04.191683 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4254a08d-4c1d-47ae-abb7-abd7ef2ae962 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:04.192316 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4254a08d-4c1d-47ae-abb7-abd7ef2ae962 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:04.192316 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4254a08d-4c1d-47ae-abb7-abd7ef2ae962 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:04.211611 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-4254a08d-4c1d-47ae-abb7-abd7ef2ae962 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:25:04.213122 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-4254a08d-4c1d-47ae-abb7-abd7ef2ae962 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:25:04.229607 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:25:04.307949 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1279/2582] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:25:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:25:04.318191 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad420044-0875-49ef-9dab-90a6e008a262 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:25:04.320284 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-ad420044-0875-49ef-9dab-90a6e008a262 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:25:04.408827 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad420044-0875-49ef-9dab-90a6e008a262 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:25:04.415369 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1309183f-e3e8-409e-9a8b-1e4b0b548f7b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '952911c492444ffea4b211a24b564117', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:25:04.416111 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1309183f-e3e8-409e-9a8b-1e4b0b548f7b service neutron] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:04.416289 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1309183f-e3e8-409e-9a8b-1e4b0b548f7b service neutron] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:04.416596 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1309183f-e3e8-409e-9a8b-1e4b0b548f7b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:04.427120 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1309183f-e3e8-409e-9a8b-1e4b0b548f7b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:25:04.429097 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1309183f-e3e8-409e-9a8b-1e4b0b548f7b service neutron] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:25:04.440600 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1304/2583] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:25:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:25:04.855078 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4254a08d-4c1d-47ae-abb7-abd7ef2ae962 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:25:04.856625 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-4254a08d-4c1d-47ae-abb7-abd7ef2ae962 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:25:04.869638 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4254a08d-4c1d-47ae-abb7-abd7ef2ae962 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:25:04.876613 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90268d46-16a8-4c67-9379-845c59266b12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:25:04.877571 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90268d46-16a8-4c67-9379-845c59266b12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:04.877853 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90268d46-16a8-4c67-9379-845c59266b12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:04.878120 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90268d46-16a8-4c67-9379-845c59266b12 admin admin] PATH_INFO: `/v3/projects/b524b2d28a5441e89749b96ec0c429fe` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:04.879399 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90268d46-16a8-4c67-9379-845c59266b12 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b524b2d28a5441e89749b96ec0c429fe)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:25:04.881327 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90268d46-16a8-4c67-9379-845c59266b12 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:25:04.935645 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-90268d46-16a8-4c67-9379-845c59266b12 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b524b2d28a5441e89749b96ec0c429fe', 'request_id': 'req-90268d46-16a8-4c67-9379-845c59266b12'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:25:04.942517 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-90268d46-16a8-4c67-9379-845c59266b12 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b524b2d28a5441e89749b96ec0c429fe', 'request_id': 'req-90268d46-16a8-4c67-9379-845c59266b12'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:25:04.948656 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-90268d46-16a8-4c67-9379-845c59266b12 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b524b2d28a5441e89749b96ec0c429fe', 'request_id': 'req-90268d46-16a8-4c67-9379-845c59266b12'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:25:04.960930 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-90268d46-16a8-4c67-9379-845c59266b12 admin admin] The token cache is being invalidate because project b524b2d28a5441e89749b96ec0c429fe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80381) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 09:25:04.961215 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-90268d46-16a8-4c67-9379-845c59266b12 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:25:04.962022 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-90268d46-16a8-4c67-9379-845c59266b12 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:25:04.963655 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1280/2584] 10.222.0.26 () {58 vars in 1232 bytes} [Thu Jul 14 09:25:04 2022] DELETE /identity/v3/projects/b524b2d28a5441e89749b96ec0c429fe => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:25:04.967862 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1309183f-e3e8-409e-9a8b-1e4b0b548f7b service neutron] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:25:04.977726 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1309183f-e3e8-409e-9a8b-1e4b0b548f7b service neutron] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:25:05.034830 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fd4626-1b2d-4eca-a7ae-2abc822d2c35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:25:05.035738 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fd4626-1b2d-4eca-a7ae-2abc822d2c35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:05.036020 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fd4626-1b2d-4eca-a7ae-2abc822d2c35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:05.036332 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fd4626-1b2d-4eca-a7ae-2abc822d2c35 admin admin] PATH_INFO: `/v3/users/9b34aeda4eda47e49b359c8658e89e58` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:05.037558 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fd4626-1b2d-4eca-a7ae-2abc822d2c35 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9b34aeda4eda47e49b359c8658e89e58)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:25:05.039679 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fd4626-1b2d-4eca-a7ae-2abc822d2c35 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:25:05.098309 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-31fd4626-1b2d-4eca-a7ae-2abc822d2c35 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9b34aeda4eda47e49b359c8658e89e58'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:25:05.098961 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-31fd4626-1b2d-4eca-a7ae-2abc822d2c35 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9b34aeda4eda47e49b359c8658e89e58'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:25:05.110857 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-31fd4626-1b2d-4eca-a7ae-2abc822d2c35 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9b34aeda4eda47e49b359c8658e89e58'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:25:05.119314 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-31fd4626-1b2d-4eca-a7ae-2abc822d2c35 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9b34aeda4eda47e49b359c8658e89e58'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:25:05.119993 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-31fd4626-1b2d-4eca-a7ae-2abc822d2c35 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9b34aeda4eda47e49b359c8658e89e58'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:25:05.134350 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1305/2585] 10.222.0.26 () {58 vars in 1223 bytes} [Thu Jul 14 09:25:04 2022] DELETE /identity/v3/users/9b34aeda4eda47e49b359c8658e89e58 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:25:05.764531 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90268d46-16a8-4c67-9379-845c59266b12 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:25:05.766876 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-90268d46-16a8-4c67-9379-845c59266b12 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:25:05.793986 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90268d46-16a8-4c67-9379-845c59266b12 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:25:05.899302 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8965c0c7-edae-4477-b77a-1c6af1fca64a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:25:05.900176 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8965c0c7-edae-4477-b77a-1c6af1fca64a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:05.900455 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8965c0c7-edae-4477-b77a-1c6af1fca64a admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:05.900749 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8965c0c7-edae-4477-b77a-1c6af1fca64a admin admin] PATH_INFO: `/v3/projects/d6f178168de74339bf5a482c28ad2954` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:05.901971 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8965c0c7-edae-4477-b77a-1c6af1fca64a admin admin] RBAC: Authorizing `identity:delete_project(project_id=d6f178168de74339bf5a482c28ad2954)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:25:05.903990 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8965c0c7-edae-4477-b77a-1c6af1fca64a admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:25:05.948835 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:25:05.953011 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-8965c0c7-edae-4477-b77a-1c6af1fca64a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd6f178168de74339bf5a482c28ad2954', 'request_id': 'req-8965c0c7-edae-4477-b77a-1c6af1fca64a'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:25:05.961673 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-8965c0c7-edae-4477-b77a-1c6af1fca64a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd6f178168de74339bf5a482c28ad2954', 'request_id': 'req-8965c0c7-edae-4477-b77a-1c6af1fca64a'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:25:05.968832 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-8965c0c7-edae-4477-b77a-1c6af1fca64a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd6f178168de74339bf5a482c28ad2954', 'request_id': 'req-8965c0c7-edae-4477-b77a-1c6af1fca64a'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:25:05.982615 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-8965c0c7-edae-4477-b77a-1c6af1fca64a admin admin] The token cache is being invalidate because project d6f178168de74339bf5a482c28ad2954 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80381) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 09:25:05.982902 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-8965c0c7-edae-4477-b77a-1c6af1fca64a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:25:05.983631 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-8965c0c7-edae-4477-b77a-1c6af1fca64a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:25:05.985315 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1281/2586] 10.222.0.26 () {58 vars in 1232 bytes} [Thu Jul 14 09:25:05 2022] DELETE /identity/v3/projects/d6f178168de74339bf5a482c28ad2954 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:25:05.990344 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b7bf48-3392-488f-bfd1-6043579b279b None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:05.990632 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b7bf48-3392-488f-bfd1-6043579b279b None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:05.990910 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b7bf48-3392-488f-bfd1-6043579b279b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:06.010495 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-42b7bf48-3392-488f-bfd1-6043579b279b None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:25:06.011754 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-42b7bf48-3392-488f-bfd1-6043579b279b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:25:06.022709 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:25:06.080090 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1306/2587] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:25:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:25:06.089566 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b295c53b-33ed-4a6d-8d98-06562c1fa918 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:06.089879 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b295c53b-33ed-4a6d-8d98-06562c1fa918 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:06.090158 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b295c53b-33ed-4a6d-8d98-06562c1fa918 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:06.109020 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-b295c53b-33ed-4a6d-8d98-06562c1fa918 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:25:06.110232 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-b295c53b-33ed-4a6d-8d98-06562c1fa918 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:25:06.122163 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1282/2588] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:25:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:25:06.123162 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b7bf48-3392-488f-bfd1-6043579b279b None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:25:06.133541 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b7bf48-3392-488f-bfd1-6043579b279b None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:25:06.139274 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2fca29-17e3-42c2-b7cc-ddf5385fa9bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:25:06.140045 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2fca29-17e3-42c2-b7cc-ddf5385fa9bb admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:06.140396 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2fca29-17e3-42c2-b7cc-ddf5385fa9bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:06.140743 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2fca29-17e3-42c2-b7cc-ddf5385fa9bb admin admin] PATH_INFO: `/v3/domains` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:06.141553 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2fca29-17e3-42c2-b7cc-ddf5385fa9bb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:25:06.141929 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2fca29-17e3-42c2-b7cc-ddf5385fa9bb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:25:06.143571 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2fca29-17e3-42c2-b7cc-ddf5385fa9bb admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:25:06.156509 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1307/2589] 10.222.0.26 () {58 vars in 1152 bytes} [Thu Jul 14 09:25:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:25:06.161346 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b295c53b-33ed-4a6d-8d98-06562c1fa918 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:25:06.171602 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b295c53b-33ed-4a6d-8d98-06562c1fa918 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:25:06.176363 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1810e14f-ed92-45c5-bf9d-15d5924ed8b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:25:06.177048 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1810e14f-ed92-45c5-bf9d-15d5924ed8b1 admin admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:06.177334 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1810e14f-ed92-45c5-bf9d-15d5924ed8b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:06.177610 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1810e14f-ed92-45c5-bf9d-15d5924ed8b1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:06.178322 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1810e14f-ed92-45c5-bf9d-15d5924ed8b1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:25:06.180900 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1810e14f-ed92-45c5-bf9d-15d5924ed8b1 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:25:06.208725 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1283/2590] 10.222.0.26 () {60 vars in 1152 bytes} [Thu Jul 14 09:25:06 2022] POST /identity/v3/projects => generated 383 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 09:25:06.213211 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2fca29-17e3-42c2-b7cc-ddf5385fa9bb admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:25:06.223951 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2fca29-17e3-42c2-b7cc-ddf5385fa9bb admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:25:06.282865 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf607348-267c-42ee-aa1d-7681f8b66146 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:25:06.283582 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf607348-267c-42ee-aa1d-7681f8b66146 admin admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:06.283866 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf607348-267c-42ee-aa1d-7681f8b66146 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:06.284146 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf607348-267c-42ee-aa1d-7681f8b66146 admin admin] PATH_INFO: `/v3/users` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:06.284851 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf607348-267c-42ee-aa1d-7681f8b66146 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:25:06.286865 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf607348-267c-42ee-aa1d-7681f8b66146 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:25:06.308607 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1308/2591] 10.222.0.26 () {60 vars in 1143 bytes} [Thu Jul 14 09:25:06 2022] POST /identity/v3/users => generated 292 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 09:25:06.312750 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1810e14f-ed92-45c5-bf9d-15d5924ed8b1 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:25:06.324076 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1810e14f-ed92-45c5-bf9d-15d5924ed8b1 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:25:06.329618 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b8b1db-7c44-4efa-9c70-86a5c5ad8c94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:25:06.330302 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b8b1db-7c44-4efa-9c70-86a5c5ad8c94 admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:06.330623 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b8b1db-7c44-4efa-9c70-86a5c5ad8c94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:06.330872 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b8b1db-7c44-4efa-9c70-86a5c5ad8c94 admin admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:06.331789 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b8b1db-7c44-4efa-9c70-86a5c5ad8c94 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:25:06.333410 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b8b1db-7c44-4efa-9c70-86a5c5ad8c94 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:25:06.346089 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1284/2592] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jul 14 09:25:06 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 09:25:06.350353 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf607348-267c-42ee-aa1d-7681f8b66146 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:25:06.361551 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf607348-267c-42ee-aa1d-7681f8b66146 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:25:06.366787 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9befeace-154e-44b7-96b2-7f89d87d407b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:25:06.367502 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9befeace-154e-44b7-96b2-7f89d87d407b admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:06.367793 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9befeace-154e-44b7-96b2-7f89d87d407b admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:06.368066 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9befeace-154e-44b7-96b2-7f89d87d407b admin admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:06.368997 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9befeace-154e-44b7-96b2-7f89d87d407b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:25:06.370656 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9befeace-154e-44b7-96b2-7f89d87d407b admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:25:06.381274 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1309/2593] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jul 14 09:25:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 09:25:06.385449 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b8b1db-7c44-4efa-9c70-86a5c5ad8c94 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:25:06.395824 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b8b1db-7c44-4efa-9c70-86a5c5ad8c94 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:25:06.400714 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a45813-f562-4a0b-9da8-98c364d3c6e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:25:06.401629 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a45813-f562-4a0b-9da8-98c364d3c6e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:06.401941 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a45813-f562-4a0b-9da8-98c364d3c6e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:06.402212 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a45813-f562-4a0b-9da8-98c364d3c6e6 admin admin] PATH_INFO: `/v3/projects/cd7cdbbadcc64fb985ea03b50d466528/users/70f5238434d545dab4813c32aa981e65/roles/d1691322ce6a4edab7ce722edf0f2988` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:06.417064 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a45813-f562-4a0b-9da8-98c364d3c6e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd7cdbbadcc64fb985ea03b50d466528, user_id=70f5238434d545dab4813c32aa981e65, role_id=d1691322ce6a4edab7ce722edf0f2988)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:25:06.419882 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a45813-f562-4a0b-9da8-98c364d3c6e6 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:25:06.436007 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1285/2594] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jul 14 09:25:06 2022] PUT /identity/v3/projects/cd7cdbbadcc64fb985ea03b50d466528/users/70f5238434d545dab4813c32aa981e65/roles/d1691322ce6a4edab7ce722edf0f2988 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:25:06.446771 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac79c27-d481-46db-837c-ca56ae441505 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:06.447038 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac79c27-d481-46db-837c-ca56ae441505 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:06.447335 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac79c27-d481-46db-837c-ca56ae441505 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:06.477323 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-4ac79c27-d481-46db-837c-ca56ae441505 None None] MFA Rules not processed for user `70f5238434d545dab4813c32aa981e65`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:25:06.478545 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-4ac79c27-d481-46db-837c-ca56ae441505 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:25:06.541812 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1310/2595] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:25:06 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:25:06.553584 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a45813-f562-4a0b-9da8-98c364d3c6e6 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:25:06.555460 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-98a45813-f562-4a0b-9da8-98c364d3c6e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:25:06.616086 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a45813-f562-4a0b-9da8-98c364d3c6e6 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:25:06.620850 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049cf200-ea45-48f9-915d-724e7e9ec81f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e76f1952a5b146808844b9f2b7395035', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:25:06.621538 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049cf200-ea45-48f9-915d-724e7e9ec81f service cinder] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:06.621824 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049cf200-ea45-48f9-915d-724e7e9ec81f service cinder] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:06.622096 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049cf200-ea45-48f9-915d-724e7e9ec81f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:06.631622 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049cf200-ea45-48f9-915d-724e7e9ec81f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:25:06.633053 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049cf200-ea45-48f9-915d-724e7e9ec81f service cinder] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:25:06.642604 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1286/2596] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:25:06 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:25:10.657596 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac79c27-d481-46db-837c-ca56ae441505 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:25:10.668400 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac79c27-d481-46db-837c-ca56ae441505 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:25:10.672853 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df0ad0b-760c-40f1-b471-03ad57bbed33 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e76f1952a5b146808844b9f2b7395035', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:25:10.673504 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df0ad0b-760c-40f1-b471-03ad57bbed33 service cinder] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:10.673775 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df0ad0b-760c-40f1-b471-03ad57bbed33 service cinder] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:10.674047 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df0ad0b-760c-40f1-b471-03ad57bbed33 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:10.676076 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-3df0ad0b-760c-40f1-b471-03ad57bbed33 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:25:10.682377 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:25:10.739445 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df0ad0b-760c-40f1-b471-03ad57bbed33 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:25:10.740883 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df0ad0b-760c-40f1-b471-03ad57bbed33 service cinder] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:25:10.750698 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1311/2597] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:25:10 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:25:15.404209 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049cf200-ea45-48f9-915d-724e7e9ec81f service cinder] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:25:15.405842 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-049cf200-ea45-48f9-915d-724e7e9ec81f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:25:15.498538 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049cf200-ea45-48f9-915d-724e7e9ec81f service cinder] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:25:15.503089 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674040cc-0c42-486b-a1d2-8cf198955fc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:25:15.504031 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674040cc-0c42-486b-a1d2-8cf198955fc6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:15.504315 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674040cc-0c42-486b-a1d2-8cf198955fc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:15.504621 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674040cc-0c42-486b-a1d2-8cf198955fc6 admin admin] PATH_INFO: `/v3/users/b5e7d732408140dcb3eba54d165b18ce` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:15.505938 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674040cc-0c42-486b-a1d2-8cf198955fc6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b5e7d732408140dcb3eba54d165b18ce)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:25:15.508168 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674040cc-0c42-486b-a1d2-8cf198955fc6 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:25:15.523521 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:25:15.564517 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-674040cc-0c42-486b-a1d2-8cf198955fc6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b5e7d732408140dcb3eba54d165b18ce'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:25:15.573009 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-674040cc-0c42-486b-a1d2-8cf198955fc6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b5e7d732408140dcb3eba54d165b18ce'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:25:15.573623 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-674040cc-0c42-486b-a1d2-8cf198955fc6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b5e7d732408140dcb3eba54d165b18ce'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:25:15.574345 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-674040cc-0c42-486b-a1d2-8cf198955fc6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b5e7d732408140dcb3eba54d165b18ce'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:25:15.581430 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-674040cc-0c42-486b-a1d2-8cf198955fc6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b5e7d732408140dcb3eba54d165b18ce'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:25:15.599524 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1287/2598] 10.222.0.26 () {58 vars in 1223 bytes} [Thu Jul 14 09:25:15 2022] DELETE /identity/v3/users/b5e7d732408140dcb3eba54d165b18ce => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:25:15.604251 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d235f3e-719d-4ec1-b14e-c5a22242caf0 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:15.604759 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d235f3e-719d-4ec1-b14e-c5a22242caf0 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:15.604862 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d235f3e-719d-4ec1-b14e-c5a22242caf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:15.623076 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-0d235f3e-719d-4ec1-b14e-c5a22242caf0 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:25:15.624334 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-0d235f3e-719d-4ec1-b14e-c5a22242caf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:25:15.692158 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1312/2599] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:25:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:25:15.701917 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674040cc-0c42-486b-a1d2-8cf198955fc6 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:25:15.703373 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-674040cc-0c42-486b-a1d2-8cf198955fc6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:25:15.763130 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674040cc-0c42-486b-a1d2-8cf198955fc6 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:25:15.766841 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98042134-475e-43d7-895f-cc7e3fe375e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '952911c492444ffea4b211a24b564117', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:25:15.767532 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98042134-475e-43d7-895f-cc7e3fe375e0 service neutron] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:15.767820 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98042134-475e-43d7-895f-cc7e3fe375e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:15.768097 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98042134-475e-43d7-895f-cc7e3fe375e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:15.778479 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98042134-475e-43d7-895f-cc7e3fe375e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:25:15.779879 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98042134-475e-43d7-895f-cc7e3fe375e0 service neutron] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:25:15.790263 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1288/2600] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:25:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:25:16.097085 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d235f3e-719d-4ec1-b14e-c5a22242caf0 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:25:16.102534 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-0d235f3e-719d-4ec1-b14e-c5a22242caf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:25:16.131580 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d235f3e-719d-4ec1-b14e-c5a22242caf0 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:25:16.139703 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacc8649-f7c9-4c87-9c08-f6b859336c42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:25:16.141209 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacc8649-f7c9-4c87-9c08-f6b859336c42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:16.141673 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98042134-475e-43d7-895f-cc7e3fe375e0 service neutron] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:25:16.141836 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacc8649-f7c9-4c87-9c08-f6b859336c42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:16.142279 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacc8649-f7c9-4c87-9c08-f6b859336c42 admin admin] PATH_INFO: `/v3/projects/65793779a11d41d893de762d062dc422` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:16.143305 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-98042134-475e-43d7-895f-cc7e3fe375e0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:25:16.144401 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacc8649-f7c9-4c87-9c08-f6b859336c42 admin admin] RBAC: Authorizing `identity:delete_project(project_id=65793779a11d41d893de762d062dc422)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:25:16.147315 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacc8649-f7c9-4c87-9c08-f6b859336c42 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:25:16.161237 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98042134-475e-43d7-895f-cc7e3fe375e0 service neutron] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:25:16.167806 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68664b89-c922-4467-8db7-2c35a4c92b49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:25:16.168811 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68664b89-c922-4467-8db7-2c35a4c92b49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:16.169393 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68664b89-c922-4467-8db7-2c35a4c92b49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:16.169775 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68664b89-c922-4467-8db7-2c35a4c92b49 admin admin] PATH_INFO: `/v3/users/70f5238434d545dab4813c32aa981e65` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:16.171080 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68664b89-c922-4467-8db7-2c35a4c92b49 admin admin] RBAC: Authorizing `identity:delete_user(user_id=70f5238434d545dab4813c32aa981e65)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:25:16.173292 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68664b89-c922-4467-8db7-2c35a4c92b49 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:25:16.227152 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-aacc8649-f7c9-4c87-9c08-f6b859336c42 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '65793779a11d41d893de762d062dc422', 'request_id': 'req-aacc8649-f7c9-4c87-9c08-f6b859336c42'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:25:16.236980 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-aacc8649-f7c9-4c87-9c08-f6b859336c42 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '65793779a11d41d893de762d062dc422', 'request_id': 'req-aacc8649-f7c9-4c87-9c08-f6b859336c42'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:25:16.239610 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-68664b89-c922-4467-8db7-2c35a4c92b49 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '70f5238434d545dab4813c32aa981e65'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:25:16.246462 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-aacc8649-f7c9-4c87-9c08-f6b859336c42 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '65793779a11d41d893de762d062dc422', 'request_id': 'req-aacc8649-f7c9-4c87-9c08-f6b859336c42'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:25:16.256405 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-68664b89-c922-4467-8db7-2c35a4c92b49 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '70f5238434d545dab4813c32aa981e65'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:25:16.257049 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-68664b89-c922-4467-8db7-2c35a4c92b49 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '70f5238434d545dab4813c32aa981e65'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:25:16.257808 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-68664b89-c922-4467-8db7-2c35a4c92b49 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '70f5238434d545dab4813c32aa981e65'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:25:16.267510 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-68664b89-c922-4467-8db7-2c35a4c92b49 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '70f5238434d545dab4813c32aa981e65'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:25:16.268622 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-aacc8649-f7c9-4c87-9c08-f6b859336c42 admin admin] The token cache is being invalidate because project 65793779a11d41d893de762d062dc422 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80382) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 09:25:16.271334 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-aacc8649-f7c9-4c87-9c08-f6b859336c42 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:25:16.272284 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-aacc8649-f7c9-4c87-9c08-f6b859336c42 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:25:16.277066 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1313/2601] 10.222.0.26 () {58 vars in 1232 bytes} [Thu Jul 14 09:25:16 2022] DELETE /identity/v3/projects/65793779a11d41d893de762d062dc422 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:25:16.278164 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacc8649-f7c9-4c87-9c08-f6b859336c42 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:25:16.280026 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-aacc8649-f7c9-4c87-9c08-f6b859336c42 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:25:16.304043 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1289/2602] 10.222.0.26 () {58 vars in 1223 bytes} [Thu Jul 14 09:25:16 2022] DELETE /identity/v3/users/70f5238434d545dab4813c32aa981e65 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 09:25:16.313047 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c307e36-56c0-4667-b448-72bd9bc47bbd None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:16.313426 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c307e36-56c0-4667-b448-72bd9bc47bbd None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:16.313775 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c307e36-56c0-4667-b448-72bd9bc47bbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:16.323719 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:25:16.342858 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-4c307e36-56c0-4667-b448-72bd9bc47bbd None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:25:16.346433 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-4c307e36-56c0-4667-b448-72bd9bc47bbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:25:16.359609 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:25:16.409535 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacc8649-f7c9-4c87-9c08-f6b859336c42 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:25:16.449606 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1290/2603] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:25:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:25:16.482298 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c307e36-56c0-4667-b448-72bd9bc47bbd None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:25:16.484152 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-4c307e36-56c0-4667-b448-72bd9bc47bbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:25:16.543346 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73744d2f-7ab4-4dd7-b66d-1fbc493e94ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:25:16.544883 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73744d2f-7ab4-4dd7-b66d-1fbc493e94ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:16.545357 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73744d2f-7ab4-4dd7-b66d-1fbc493e94ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:16.545830 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73744d2f-7ab4-4dd7-b66d-1fbc493e94ea admin admin] PATH_INFO: `/v3/users/8d192883bb6c49d989a203a0004aa7bc` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:16.548316 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73744d2f-7ab4-4dd7-b66d-1fbc493e94ea admin admin] RBAC: Authorizing `identity:delete_user(user_id=8d192883bb6c49d989a203a0004aa7bc)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:25:16.551431 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73744d2f-7ab4-4dd7-b66d-1fbc493e94ea admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:25:16.574620 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c307e36-56c0-4667-b448-72bd9bc47bbd None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:25:16.579964 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee9f2b7-e27d-41f2-84f3-5acca66fa526 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '952911c492444ffea4b211a24b564117', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:25:16.580746 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee9f2b7-e27d-41f2-84f3-5acca66fa526 service neutron] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:16.581347 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee9f2b7-e27d-41f2-84f3-5acca66fa526 service neutron] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:16.581532 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee9f2b7-e27d-41f2-84f3-5acca66fa526 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:16.593080 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee9f2b7-e27d-41f2-84f3-5acca66fa526 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:25:16.595045 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee9f2b7-e27d-41f2-84f3-5acca66fa526 service neutron] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:25:16.607525 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1291/2604] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:25:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 09:25:16.624890 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-73744d2f-7ab4-4dd7-b66d-1fbc493e94ea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8d192883bb6c49d989a203a0004aa7bc'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:25:16.625913 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-73744d2f-7ab4-4dd7-b66d-1fbc493e94ea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8d192883bb6c49d989a203a0004aa7bc'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:25:16.640993 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-73744d2f-7ab4-4dd7-b66d-1fbc493e94ea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8d192883bb6c49d989a203a0004aa7bc'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:25:16.650535 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-73744d2f-7ab4-4dd7-b66d-1fbc493e94ea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8d192883bb6c49d989a203a0004aa7bc'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:25:16.651667 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-73744d2f-7ab4-4dd7-b66d-1fbc493e94ea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8d192883bb6c49d989a203a0004aa7bc'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:25:16.666175 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1314/2605] 10.222.0.26 () {58 vars in 1223 bytes} [Thu Jul 14 09:25:16 2022] DELETE /identity/v3/users/8d192883bb6c49d989a203a0004aa7bc => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:25:17.249556 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee9f2b7-e27d-41f2-84f3-5acca66fa526 service neutron] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:25:17.251549 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-3ee9f2b7-e27d-41f2-84f3-5acca66fa526 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:25:17.266469 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73744d2f-7ab4-4dd7-b66d-1fbc493e94ea admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:25:17.268063 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-73744d2f-7ab4-4dd7-b66d-1fbc493e94ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:25:17.281803 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee9f2b7-e27d-41f2-84f3-5acca66fa526 service neutron] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:25:17.350786 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:25:17.355797 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73744d2f-7ab4-4dd7-b66d-1fbc493e94ea admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:25:17.360375 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb5f849-2ebc-4845-91b7-e12322518b26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:25:17.361781 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb5f849-2ebc-4845-91b7-e12322518b26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:17.362076 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb5f849-2ebc-4845-91b7-e12322518b26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:17.362385 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb5f849-2ebc-4845-91b7-e12322518b26 admin admin] PATH_INFO: `/v3/projects/cd7cdbbadcc64fb985ea03b50d466528` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:17.362970 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5af8d12-8cb4-4167-9468-52109f5f813b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:25:17.364248 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5af8d12-8cb4-4167-9468-52109f5f813b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:17.364372 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb5f849-2ebc-4845-91b7-e12322518b26 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cd7cdbbadcc64fb985ea03b50d466528)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:25:17.364799 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5af8d12-8cb4-4167-9468-52109f5f813b admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:17.365254 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5af8d12-8cb4-4167-9468-52109f5f813b admin admin] PATH_INFO: `/v3/projects/410d02e8ff244020ad902bf578383bc9` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:17.366305 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb5f849-2ebc-4845-91b7-e12322518b26 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:25:17.367031 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5af8d12-8cb4-4167-9468-52109f5f813b admin admin] RBAC: Authorizing `identity:delete_project(project_id=410d02e8ff244020ad902bf578383bc9)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:25:17.369454 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5af8d12-8cb4-4167-9468-52109f5f813b admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:25:17.422492 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-d5af8d12-8cb4-4167-9468-52109f5f813b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '410d02e8ff244020ad902bf578383bc9', 'request_id': 'req-d5af8d12-8cb4-4167-9468-52109f5f813b'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:25:17.429194 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-aeb5f849-2ebc-4845-91b7-e12322518b26 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cd7cdbbadcc64fb985ea03b50d466528', 'request_id': 'req-aeb5f849-2ebc-4845-91b7-e12322518b26'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:25:17.430408 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:25:17.442746 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-d5af8d12-8cb4-4167-9468-52109f5f813b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '410d02e8ff244020ad902bf578383bc9', 'request_id': 'req-d5af8d12-8cb4-4167-9468-52109f5f813b'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:25:17.445485 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-aeb5f849-2ebc-4845-91b7-e12322518b26 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cd7cdbbadcc64fb985ea03b50d466528', 'request_id': 'req-aeb5f849-2ebc-4845-91b7-e12322518b26'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:25:17.457110 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-d5af8d12-8cb4-4167-9468-52109f5f813b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '410d02e8ff244020ad902bf578383bc9', 'request_id': 'req-d5af8d12-8cb4-4167-9468-52109f5f813b'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:25:17.462266 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-aeb5f849-2ebc-4845-91b7-e12322518b26 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cd7cdbbadcc64fb985ea03b50d466528', 'request_id': 'req-aeb5f849-2ebc-4845-91b7-e12322518b26'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:25:17.475158 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-d5af8d12-8cb4-4167-9468-52109f5f813b admin admin] The token cache is being invalidate because project 410d02e8ff244020ad902bf578383bc9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80382) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 09:25:17.475548 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-d5af8d12-8cb4-4167-9468-52109f5f813b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:25:17.476479 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-d5af8d12-8cb4-4167-9468-52109f5f813b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:25:17.481757 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1315/2606] 10.222.0.26 () {58 vars in 1232 bytes} [Thu Jul 14 09:25:17 2022] DELETE /identity/v3/projects/410d02e8ff244020ad902bf578383bc9 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:25:17.483234 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb2eb6f-b5c5-48e2-94c0-52a5c94cbe2b None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:17.483564 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-aeb5f849-2ebc-4845-91b7-e12322518b26 admin admin] The token cache is being invalidate because project cd7cdbbadcc64fb985ea03b50d466528 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80381) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 09:25:17.483689 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb2eb6f-b5c5-48e2-94c0-52a5c94cbe2b None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:17.483858 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-aeb5f849-2ebc-4845-91b7-e12322518b26 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:25:17.483993 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb2eb6f-b5c5-48e2-94c0-52a5c94cbe2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:17.484510 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-aeb5f849-2ebc-4845-91b7-e12322518b26 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:25:17.486552 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1292/2607] 10.222.0.26 () {58 vars in 1232 bytes} [Thu Jul 14 09:25:17 2022] DELETE /identity/v3/projects/cd7cdbbadcc64fb985ea03b50d466528 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:25:17.492315 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd0ba8e-a963-427f-b99a-bdfce8216d3a None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:17.492687 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd0ba8e-a963-427f-b99a-bdfce8216d3a None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:17.493057 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd0ba8e-a963-427f-b99a-bdfce8216d3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:17.517434 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-1cd0ba8e-a963-427f-b99a-bdfce8216d3a None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:25:17.519119 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-1cd0ba8e-a963-427f-b99a-bdfce8216d3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:25:17.566259 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-ceb2eb6f-b5c5-48e2-94c0-52a5c94cbe2b None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:25:17.571002 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-ceb2eb6f-b5c5-48e2-94c0-52a5c94cbe2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:25:17.646503 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1293/2608] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:25:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:25:17.651275 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04e85eb-41cc-43db-8b1e-4845d4d3320a None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:17.651565 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04e85eb-41cc-43db-8b1e-4845d4d3320a None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:17.651877 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04e85eb-41cc-43db-8b1e-4845d4d3320a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:17.674427 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1316/2609] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:25:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:25:17.680366 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-e04e85eb-41cc-43db-8b1e-4845d4d3320a None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:25:17.680958 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da051ab5-54d2-4751-8cfa-d80e1e64e067 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:17.681641 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da051ab5-54d2-4751-8cfa-d80e1e64e067 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:17.681768 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-e04e85eb-41cc-43db-8b1e-4845d4d3320a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:25:17.682175 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da051ab5-54d2-4751-8cfa-d80e1e64e067 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:17.695484 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1294/2610] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:25:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:25:17.696373 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04e85eb-41cc-43db-8b1e-4845d4d3320a None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:25:17.706773 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04e85eb-41cc-43db-8b1e-4845d4d3320a None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:25:17.712203 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937d189a-19a5-42dd-9340-af7bb3101887 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:25:17.715226 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937d189a-19a5-42dd-9340-af7bb3101887 admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:17.715226 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937d189a-19a5-42dd-9340-af7bb3101887 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:17.715226 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937d189a-19a5-42dd-9340-af7bb3101887 admin admin] PATH_INFO: `/v3/domains` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:17.715226 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937d189a-19a5-42dd-9340-af7bb3101887 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:25:17.715226 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937d189a-19a5-42dd-9340-af7bb3101887 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:25:17.716209 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937d189a-19a5-42dd-9340-af7bb3101887 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:25:17.728605 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-da051ab5-54d2-4751-8cfa-d80e1e64e067 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:25:17.730627 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-da051ab5-54d2-4751-8cfa-d80e1e64e067 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:25:17.737540 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1295/2611] 10.222.0.26 () {58 vars in 1152 bytes} [Thu Jul 14 09:25:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:25:17.738398 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937d189a-19a5-42dd-9340-af7bb3101887 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:25:17.741678 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1317/2612] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:25:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:25:17.748859 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da051ab5-54d2-4751-8cfa-d80e1e64e067 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:25:17.752359 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937d189a-19a5-42dd-9340-af7bb3101887 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:25:17.758775 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7dec1e-b6be-4a16-b0f4-963f9cffbb7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:25:17.759561 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7dec1e-b6be-4a16-b0f4-963f9cffbb7f admin admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:17.759844 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7dec1e-b6be-4a16-b0f4-963f9cffbb7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:17.760117 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7dec1e-b6be-4a16-b0f4-963f9cffbb7f admin admin] PATH_INFO: `/v3/projects` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:17.760892 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7dec1e-b6be-4a16-b0f4-963f9cffbb7f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:25:17.763616 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7dec1e-b6be-4a16-b0f4-963f9cffbb7f admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:25:17.764888 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da051ab5-54d2-4751-8cfa-d80e1e64e067 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:25:17.771751 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3e6aad-9c95-4449-a28e-3be0a27e5dad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:25:17.772613 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3e6aad-9c95-4449-a28e-3be0a27e5dad admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:17.772958 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3e6aad-9c95-4449-a28e-3be0a27e5dad admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:17.773373 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3e6aad-9c95-4449-a28e-3be0a27e5dad admin admin] PATH_INFO: `/v3/domains` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:17.774242 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3e6aad-9c95-4449-a28e-3be0a27e5dad admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:25:17.774635 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3e6aad-9c95-4449-a28e-3be0a27e5dad admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:25:17.776435 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3e6aad-9c95-4449-a28e-3be0a27e5dad admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:25:17.789562 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1318/2613] 10.222.0.26 () {58 vars in 1152 bytes} [Thu Jul 14 09:25:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:25:17.794167 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3e6aad-9c95-4449-a28e-3be0a27e5dad admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:25:17.804813 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3e6aad-9c95-4449-a28e-3be0a27e5dad admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:25:17.809749 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1296/2614] 10.222.0.26 () {60 vars in 1152 bytes} [Thu Jul 14 09:25:17 2022] POST /identity/v3/projects => generated 369 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 09:25:17.811530 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761e0366-56d7-48ed-8a0f-9fae11e87e0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:25:17.812367 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761e0366-56d7-48ed-8a0f-9fae11e87e0a admin admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:17.812594 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7dec1e-b6be-4a16-b0f4-963f9cffbb7f admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:25:17.812856 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761e0366-56d7-48ed-8a0f-9fae11e87e0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:17.813248 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761e0366-56d7-48ed-8a0f-9fae11e87e0a admin admin] PATH_INFO: `/v3/projects` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:17.814059 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761e0366-56d7-48ed-8a0f-9fae11e87e0a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:25:17.816304 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761e0366-56d7-48ed-8a0f-9fae11e87e0a admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:25:17.848508 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1319/2615] 10.222.0.26 () {60 vars in 1152 bytes} [Thu Jul 14 09:25:17 2022] POST /identity/v3/projects => generated 365 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 09:25:17.850379 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761e0366-56d7-48ed-8a0f-9fae11e87e0a admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:25:17.884983 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7dec1e-b6be-4a16-b0f4-963f9cffbb7f admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:25:17.905683 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:25:17.926557 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761e0366-56d7-48ed-8a0f-9fae11e87e0a admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:25:17.937979 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=95,keystone:DELETE=3,keystone:INSERT=2 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:25:17.938362 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a0d5f1-4b1a-4a68-921e-21d80d4c7d14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:25:17.940191 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a0d5f1-4b1a-4a68-921e-21d80d4c7d14 admin admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:17.940191 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a0d5f1-4b1a-4a68-921e-21d80d4c7d14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:17.940191 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a0d5f1-4b1a-4a68-921e-21d80d4c7d14 admin admin] PATH_INFO: `/v3/users` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:17.941212 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a0d5f1-4b1a-4a68-921e-21d80d4c7d14 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:25:17.943539 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a0d5f1-4b1a-4a68-921e-21d80d4c7d14 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:25:17.959516 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a84009-c6fd-452a-a323-36cc613dc640 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:25:17.960195 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a84009-c6fd-452a-a323-36cc613dc640 admin admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:17.960481 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a84009-c6fd-452a-a323-36cc613dc640 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:17.960758 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a84009-c6fd-452a-a323-36cc613dc640 admin admin] PATH_INFO: `/v3/users` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:17.961523 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a84009-c6fd-452a-a323-36cc613dc640 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:25:17.963635 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a84009-c6fd-452a-a323-36cc613dc640 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:25:17.973227 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1320/2616] 10.222.0.26 () {60 vars in 1143 bytes} [Thu Jul 14 09:25:17 2022] POST /identity/v3/users => generated 283 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 09:25:17.979903 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a0d5f1-4b1a-4a68-921e-21d80d4c7d14 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:25:17.988378 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1297/2617] 10.222.0.26 () {60 vars in 1143 bytes} [Thu Jul 14 09:25:17 2022] POST /identity/v3/users => generated 285 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 09:25:17.992368 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a84009-c6fd-452a-a323-36cc613dc640 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:25:17.993363 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a0d5f1-4b1a-4a68-921e-21d80d4c7d14 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:25:18.001460 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d786387-e1ea-4a6e-a944-da40c7bbced7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:25:18.002309 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d786387-e1ea-4a6e-a944-da40c7bbced7 admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:18.002977 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d786387-e1ea-4a6e-a944-da40c7bbced7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:18.003235 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d786387-e1ea-4a6e-a944-da40c7bbced7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:18.004337 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d786387-e1ea-4a6e-a944-da40c7bbced7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:25:18.005154 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a84009-c6fd-452a-a323-36cc613dc640 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:25:18.005956 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d786387-e1ea-4a6e-a944-da40c7bbced7 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:25:18.010621 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428e281c-aef0-4904-9a3c-7325e04f923c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:25:18.011436 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428e281c-aef0-4904-9a3c-7325e04f923c admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:18.011802 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428e281c-aef0-4904-9a3c-7325e04f923c admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:18.012222 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428e281c-aef0-4904-9a3c-7325e04f923c admin admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:18.012811 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428e281c-aef0-4904-9a3c-7325e04f923c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:25:18.014843 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428e281c-aef0-4904-9a3c-7325e04f923c admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:25:18.024482 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1321/2618] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jul 14 09:25:17 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 09:25:18.025837 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d786387-e1ea-4a6e-a944-da40c7bbced7 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:25:18.028486 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1298/2619] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jul 14 09:25:17 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 09:25:18.032358 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428e281c-aef0-4904-9a3c-7325e04f923c admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:25:18.037155 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d786387-e1ea-4a6e-a944-da40c7bbced7 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:25:18.042174 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d679ddee-be32-4214-b794-84875c963be6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:25:18.043215 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d679ddee-be32-4214-b794-84875c963be6 admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:18.043356 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428e281c-aef0-4904-9a3c-7325e04f923c admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:25:18.043450 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d679ddee-be32-4214-b794-84875c963be6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:18.043553 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d679ddee-be32-4214-b794-84875c963be6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:18.044333 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d679ddee-be32-4214-b794-84875c963be6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:25:18.046109 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d679ddee-be32-4214-b794-84875c963be6 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:25:18.049983 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2463b91f-0a60-4cd3-b0cd-5de615f0489e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:25:18.050650 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2463b91f-0a60-4cd3-b0cd-5de615f0489e admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:18.050931 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2463b91f-0a60-4cd3-b0cd-5de615f0489e admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:18.051296 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2463b91f-0a60-4cd3-b0cd-5de615f0489e admin admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:18.051959 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2463b91f-0a60-4cd3-b0cd-5de615f0489e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:25:18.053583 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2463b91f-0a60-4cd3-b0cd-5de615f0489e admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:25:18.063500 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1322/2620] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jul 14 09:25:18 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 09:25:18.064784 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d679ddee-be32-4214-b794-84875c963be6 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:25:18.070250 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1299/2621] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jul 14 09:25:18 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 09:25:18.071138 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2463b91f-0a60-4cd3-b0cd-5de615f0489e admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:25:18.077132 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d679ddee-be32-4214-b794-84875c963be6 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:25:18.081464 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2463b91f-0a60-4cd3-b0cd-5de615f0489e admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:25:18.082990 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab19bbd4-0373-431d-a7e2-5187e38972f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:25:18.084020 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab19bbd4-0373-431d-a7e2-5187e38972f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:18.084360 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab19bbd4-0373-431d-a7e2-5187e38972f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:18.084631 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab19bbd4-0373-431d-a7e2-5187e38972f4 admin admin] PATH_INFO: `/v3/projects/0bcc0bfc95694837a3feb523b77e6eb3/users/ba01a495640d4ab2803bbc1872e94dbc/roles/d1691322ce6a4edab7ce722edf0f2988` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:18.088356 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319d1c47-7aec-4d87-b186-e5badbe04a85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:25:18.089571 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319d1c47-7aec-4d87-b186-e5badbe04a85 admin admin] REQUEST_METHOD: `PUT` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:18.089940 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319d1c47-7aec-4d87-b186-e5badbe04a85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:18.090400 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319d1c47-7aec-4d87-b186-e5badbe04a85 admin admin] PATH_INFO: `/v3/projects/c58b6f27d77e4223b86fc424e4b21f9b/users/5b511b3f04d54edd8e694431c77b0e1b/roles/d1691322ce6a4edab7ce722edf0f2988` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:18.101410 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab19bbd4-0373-431d-a7e2-5187e38972f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0bcc0bfc95694837a3feb523b77e6eb3, user_id=ba01a495640d4ab2803bbc1872e94dbc, role_id=d1691322ce6a4edab7ce722edf0f2988)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:25:18.104725 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab19bbd4-0373-431d-a7e2-5187e38972f4 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:25:18.108953 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319d1c47-7aec-4d87-b186-e5badbe04a85 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c58b6f27d77e4223b86fc424e4b21f9b, user_id=5b511b3f04d54edd8e694431c77b0e1b, role_id=d1691322ce6a4edab7ce722edf0f2988)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:25:18.112131 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319d1c47-7aec-4d87-b186-e5badbe04a85 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:25:18.122448 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1323/2622] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jul 14 09:25:18 2022] PUT /identity/v3/projects/0bcc0bfc95694837a3feb523b77e6eb3/users/ba01a495640d4ab2803bbc1872e94dbc/roles/d1691322ce6a4edab7ce722edf0f2988 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:25:18.133023 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514f3f1b-47ff-4f51-863c-d04924e1e0d0 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:18.133023 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514f3f1b-47ff-4f51-863c-d04924e1e0d0 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:18.133349 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514f3f1b-47ff-4f51-863c-d04924e1e0d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:18.141695 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1300/2623] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jul 14 09:25:18 2022] PUT /identity/v3/projects/c58b6f27d77e4223b86fc424e4b21f9b/users/5b511b3f04d54edd8e694431c77b0e1b/roles/d1691322ce6a4edab7ce722edf0f2988 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:25:18.169792 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-514f3f1b-47ff-4f51-863c-d04924e1e0d0 None None] MFA Rules not processed for user `ba01a495640d4ab2803bbc1872e94dbc`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:25:18.171063 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-514f3f1b-47ff-4f51-863c-d04924e1e0d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:25:18.208141 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01e0cb3-0cb5-49bb-bf35-6027f6dd66c7 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:18.208344 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01e0cb3-0cb5-49bb-bf35-6027f6dd66c7 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:18.208606 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01e0cb3-0cb5-49bb-bf35-6027f6dd66c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:18.245044 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-f01e0cb3-0cb5-49bb-bf35-6027f6dd66c7 None None] MFA Rules not processed for user `5b511b3f04d54edd8e694431c77b0e1b`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:25:18.246366 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-f01e0cb3-0cb5-49bb-bf35-6027f6dd66c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:25:18.250040 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1324/2624] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:25:18 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 09:25:18.261218 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514f3f1b-47ff-4f51-863c-d04924e1e0d0 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:25:18.343319 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1301/2625] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:25:18 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 09:25:18.351674 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514f3f1b-47ff-4f51-863c-d04924e1e0d0 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:25:18.354615 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01e0cb3-0cb5-49bb-bf35-6027f6dd66c7 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:25:18.356051 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-f01e0cb3-0cb5-49bb-bf35-6027f6dd66c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:25:18.356578 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868eb480-24d9-42fe-bee1-414caab5a0c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:25:18.357250 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868eb480-24d9-42fe-bee1-414caab5a0c9 admin admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:18.357535 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868eb480-24d9-42fe-bee1-414caab5a0c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:18.357936 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868eb480-24d9-42fe-bee1-414caab5a0c9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:18.358646 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868eb480-24d9-42fe-bee1-414caab5a0c9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:25:18.360608 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868eb480-24d9-42fe-bee1-414caab5a0c9 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:25:18.391578 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1325/2626] 10.222.0.26 () {60 vars in 1152 bytes} [Thu Jul 14 09:25:18 2022] POST /identity/v3/projects => generated 363 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 09:25:18.392597 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868eb480-24d9-42fe-bee1-414caab5a0c9 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:25:18.400408 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:25:18.422241 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01e0cb3-0cb5-49bb-bf35-6027f6dd66c7 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:25:18.466205 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868eb480-24d9-42fe-bee1-414caab5a0c9 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:25:18.471573 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97dddabd-bfff-4351-85d3-9570a2cf1f6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:25:18.472269 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dddabd-bfff-4351-85d3-9570a2cf1f6a admin admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:18.472553 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dddabd-bfff-4351-85d3-9570a2cf1f6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:18.472846 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dddabd-bfff-4351-85d3-9570a2cf1f6a admin admin] PATH_INFO: `/v3/users` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:18.473535 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97dddabd-bfff-4351-85d3-9570a2cf1f6a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:25:18.475634 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97dddabd-bfff-4351-85d3-9570a2cf1f6a admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:25:18.476203 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:25:18.503376 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1326/2627] 10.222.0.26 () {60 vars in 1143 bytes} [Thu Jul 14 09:25:18 2022] POST /identity/v3/users => generated 288 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 09:25:18.508792 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97dddabd-bfff-4351-85d3-9570a2cf1f6a admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:25:18.513499 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285508f9-715e-4d06-8f9f-08075ad0e656 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '872dbb91c9e54f13b1a31fc8242126df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:25:18.514161 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285508f9-715e-4d06-8f9f-08075ad0e656 service glance] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:18.514443 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285508f9-715e-4d06-8f9f-08075ad0e656 service glance] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:18.514720 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285508f9-715e-4d06-8f9f-08075ad0e656 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:18.525454 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97dddabd-bfff-4351-85d3-9570a2cf1f6a admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:25:18.533267 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b3ee37-6018-44f6-a11a-5ba1c031ec90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:25:18.534498 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b3ee37-6018-44f6-a11a-5ba1c031ec90 admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:18.534904 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b3ee37-6018-44f6-a11a-5ba1c031ec90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:18.535299 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b3ee37-6018-44f6-a11a-5ba1c031ec90 admin admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:18.536316 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b3ee37-6018-44f6-a11a-5ba1c031ec90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:25:18.538112 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b3ee37-6018-44f6-a11a-5ba1c031ec90 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:25:18.551753 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1327/2628] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jul 14 09:25:18 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 09:25:18.555159 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b3ee37-6018-44f6-a11a-5ba1c031ec90 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:25:18.568078 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b3ee37-6018-44f6-a11a-5ba1c031ec90 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:25:18.588424 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d62cfc-3e00-4500-8df8-19a8040250f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:25:18.589553 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d62cfc-3e00-4500-8df8-19a8040250f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:18.589836 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d62cfc-3e00-4500-8df8-19a8040250f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:18.590291 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d62cfc-3e00-4500-8df8-19a8040250f9 admin admin] PATH_INFO: `/v3/projects/8056e60ccbac44cdb0a2b9bd66e8e745/users/468bb62f19ca45adb7183eef1481071e/roles/7b549d58d1724395b343609d895c80d0` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:18.596115 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285508f9-715e-4d06-8f9f-08075ad0e656 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:25:18.597750 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285508f9-715e-4d06-8f9f-08075ad0e656 service glance] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:25:18.616023 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1302/2629] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:25:18 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 262 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 09:25:18.620625 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d62cfc-3e00-4500-8df8-19a8040250f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8056e60ccbac44cdb0a2b9bd66e8e745, user_id=468bb62f19ca45adb7183eef1481071e, role_id=7b549d58d1724395b343609d895c80d0)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:25:18.624408 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d62cfc-3e00-4500-8df8-19a8040250f9 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:25:18.639261 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1328/2630] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jul 14 09:25:18 2022] PUT /identity/v3/projects/8056e60ccbac44cdb0a2b9bd66e8e745/users/468bb62f19ca45adb7183eef1481071e/roles/7b549d58d1724395b343609d895c80d0 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:25:18.644281 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285508f9-715e-4d06-8f9f-08075ad0e656 service glance] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:25:18.656677 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285508f9-715e-4d06-8f9f-08075ad0e656 service glance] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:25:18.729276 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79815ce6-33f8-4708-87fd-7ab857376a96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:25:18.730815 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79815ce6-33f8-4708-87fd-7ab857376a96 admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:18.732272 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79815ce6-33f8-4708-87fd-7ab857376a96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:18.732606 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79815ce6-33f8-4708-87fd-7ab857376a96 admin admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:18.733381 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79815ce6-33f8-4708-87fd-7ab857376a96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:25:18.735574 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79815ce6-33f8-4708-87fd-7ab857376a96 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:25:18.775074 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1303/2631] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jul 14 09:25:18 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 09:25:18.779821 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d62cfc-3e00-4500-8df8-19a8040250f9 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:25:18.794268 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d62cfc-3e00-4500-8df8-19a8040250f9 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:25:18.807889 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cf0210-ded7-465e-b011-2657eada8a9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:25:18.808879 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cf0210-ded7-465e-b011-2657eada8a9a admin admin] REQUEST_METHOD: `PUT` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:18.809173 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cf0210-ded7-465e-b011-2657eada8a9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:18.809447 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cf0210-ded7-465e-b011-2657eada8a9a admin admin] PATH_INFO: `/v3/projects/8056e60ccbac44cdb0a2b9bd66e8e745/users/468bb62f19ca45adb7183eef1481071e/roles/7b549d58d1724395b343609d895c80d0` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:18.816041 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cf0210-ded7-465e-b011-2657eada8a9a admin admin] RBAC: Authorizing `identity:create_grant(project_id=8056e60ccbac44cdb0a2b9bd66e8e745, user_id=468bb62f19ca45adb7183eef1481071e, role_id=7b549d58d1724395b343609d895c80d0)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:25:18.843088 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe6b48d-367e-4dc9-9b5f-9c7c17d86a15 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:18.845777 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe6b48d-367e-4dc9-9b5f-9c7c17d86a15 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:18.846684 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe6b48d-367e-4dc9-9b5f-9c7c17d86a15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:18.856337 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cf0210-ded7-465e-b011-2657eada8a9a admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:25:18.872055 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-8fe6b48d-367e-4dc9-9b5f-9c7c17d86a15 None None] MFA Rules not processed for user `e072ca213ed74cbda810a1fd971ae23d`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:25:18.874163 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-8fe6b48d-367e-4dc9-9b5f-9c7c17d86a15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:25:18.876639 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1329/2632] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jul 14 09:25:18 2022] PUT /identity/v3/projects/8056e60ccbac44cdb0a2b9bd66e8e745/users/468bb62f19ca45adb7183eef1481071e/roles/7b549d58d1724395b343609d895c80d0 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:25:18.888120 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829974ca-a776-4c69-b54c-522adc2bccac None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:18.888413 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829974ca-a776-4c69-b54c-522adc2bccac None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:18.888690 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829974ca-a776-4c69-b54c-522adc2bccac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:18.932208 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-829974ca-a776-4c69-b54c-522adc2bccac None None] MFA Rules not processed for user `468bb62f19ca45adb7183eef1481071e`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:25:18.933607 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-829974ca-a776-4c69-b54c-522adc2bccac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:25:18.972523 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1304/2633] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:25:18 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:25:19.061321 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1330/2634] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:25:18 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 09:25:19.088815 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe6b48d-367e-4dc9-9b5f-9c7c17d86a15 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:25:19.090299 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-8fe6b48d-367e-4dc9-9b5f-9c7c17d86a15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:25:19.132792 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:25:19.182700 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe6b48d-367e-4dc9-9b5f-9c7c17d86a15 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:25:19.186779 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b985135-9533-4484-a40f-269004368428 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e76f1952a5b146808844b9f2b7395035', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:25:19.188276 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b985135-9533-4484-a40f-269004368428 service cinder] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:19.188661 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b985135-9533-4484-a40f-269004368428 service cinder] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:19.189035 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b985135-9533-4484-a40f-269004368428 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:19.202421 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b985135-9533-4484-a40f-269004368428 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:25:19.204053 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b985135-9533-4484-a40f-269004368428 service cinder] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:25:19.215866 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1305/2635] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:25:19 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:25:19.319766 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829974ca-a776-4c69-b54c-522adc2bccac None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:25:19.321348 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-829974ca-a776-4c69-b54c-522adc2bccac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:25:19.342148 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829974ca-a776-4c69-b54c-522adc2bccac None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:25:19.368278 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1087053-0676-4736-ac7f-9c1aa1e28735 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e76f1952a5b146808844b9f2b7395035', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:25:19.370291 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1087053-0676-4736-ac7f-9c1aa1e28735 service cinder] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:19.371600 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1087053-0676-4736-ac7f-9c1aa1e28735 service cinder] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:19.374464 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1087053-0676-4736-ac7f-9c1aa1e28735 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:19.423883 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:25:19.466356 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1087053-0676-4736-ac7f-9c1aa1e28735 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:25:19.468019 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1087053-0676-4736-ac7f-9c1aa1e28735 service cinder] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:25:19.480926 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1331/2636] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:25:19 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:25:22.432970 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b985135-9533-4484-a40f-269004368428 service cinder] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:25:22.434615 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-7b985135-9533-4484-a40f-269004368428 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:25:22.600096 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b985135-9533-4484-a40f-269004368428 service cinder] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:25:22.604360 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5563b733-980f-4900-8fb6-cec4d4e1bf4b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '952911c492444ffea4b211a24b564117', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:25:22.605200 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5563b733-980f-4900-8fb6-cec4d4e1bf4b service neutron] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:22.605551 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5563b733-980f-4900-8fb6-cec4d4e1bf4b service neutron] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:22.605909 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5563b733-980f-4900-8fb6-cec4d4e1bf4b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:22.608937 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-5563b733-980f-4900-8fb6-cec4d4e1bf4b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:25:22.734724 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5563b733-980f-4900-8fb6-cec4d4e1bf4b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:25:22.736305 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5563b733-980f-4900-8fb6-cec4d4e1bf4b service neutron] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:25:22.753839 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1306/2637] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:25:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:25:26.438353 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1087053-0676-4736-ac7f-9c1aa1e28735 service cinder] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:25:26.439964 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-b1087053-0676-4736-ac7f-9c1aa1e28735 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:25:26.458087 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1087053-0676-4736-ac7f-9c1aa1e28735 service cinder] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:25:26.462867 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9dd5050-d4b9-4d69-94ac-eb4c4031cbd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:25:26.463932 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dd5050-d4b9-4d69-94ac-eb4c4031cbd6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:26.464861 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dd5050-d4b9-4d69-94ac-eb4c4031cbd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:26.465139 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dd5050-d4b9-4d69-94ac-eb4c4031cbd6 admin admin] PATH_INFO: `/v3/users/4afe1ec7d1ab4e40892d1dde44ce1667` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:26.466334 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9dd5050-d4b9-4d69-94ac-eb4c4031cbd6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4afe1ec7d1ab4e40892d1dde44ce1667)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:25:26.468615 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9dd5050-d4b9-4d69-94ac-eb4c4031cbd6 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:25:26.515474 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-d9dd5050-d4b9-4d69-94ac-eb4c4031cbd6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4afe1ec7d1ab4e40892d1dde44ce1667'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:25:26.516239 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-d9dd5050-d4b9-4d69-94ac-eb4c4031cbd6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4afe1ec7d1ab4e40892d1dde44ce1667'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:25:26.529436 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-d9dd5050-d4b9-4d69-94ac-eb4c4031cbd6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4afe1ec7d1ab4e40892d1dde44ce1667'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:25:26.538137 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-d9dd5050-d4b9-4d69-94ac-eb4c4031cbd6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4afe1ec7d1ab4e40892d1dde44ce1667'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:25:26.538950 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-d9dd5050-d4b9-4d69-94ac-eb4c4031cbd6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4afe1ec7d1ab4e40892d1dde44ce1667'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:25:26.551079 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1332/2638] 10.222.0.26 () {58 vars in 1223 bytes} [Thu Jul 14 09:25:26 2022] DELETE /identity/v3/users/4afe1ec7d1ab4e40892d1dde44ce1667 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:25:26.732917 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5563b733-980f-4900-8fb6-cec4d4e1bf4b service neutron] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:25:26.743731 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5563b733-980f-4900-8fb6-cec4d4e1bf4b service neutron] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:25:26.815550 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70fe2b5-08f8-4410-a2a4-79aaeb855d6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:25:26.816415 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70fe2b5-08f8-4410-a2a4-79aaeb855d6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:26.816703 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70fe2b5-08f8-4410-a2a4-79aaeb855d6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:26.816985 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70fe2b5-08f8-4410-a2a4-79aaeb855d6e admin admin] PATH_INFO: `/v3/projects/39418a3b1c424ed48cfc68d5ac048413` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:26.818159 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70fe2b5-08f8-4410-a2a4-79aaeb855d6e admin admin] RBAC: Authorizing `identity:delete_project(project_id=39418a3b1c424ed48cfc68d5ac048413)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:25:26.820215 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70fe2b5-08f8-4410-a2a4-79aaeb855d6e admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:25:26.844812 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:25:26.869397 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-d70fe2b5-08f8-4410-a2a4-79aaeb855d6e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '39418a3b1c424ed48cfc68d5ac048413', 'request_id': 'req-d70fe2b5-08f8-4410-a2a4-79aaeb855d6e'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:25:26.875868 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-d70fe2b5-08f8-4410-a2a4-79aaeb855d6e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '39418a3b1c424ed48cfc68d5ac048413', 'request_id': 'req-d70fe2b5-08f8-4410-a2a4-79aaeb855d6e'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:25:26.881998 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-d70fe2b5-08f8-4410-a2a4-79aaeb855d6e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '39418a3b1c424ed48cfc68d5ac048413', 'request_id': 'req-d70fe2b5-08f8-4410-a2a4-79aaeb855d6e'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:25:26.895752 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-d70fe2b5-08f8-4410-a2a4-79aaeb855d6e admin admin] The token cache is being invalidate because project 39418a3b1c424ed48cfc68d5ac048413 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80381) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 09:25:26.896051 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-d70fe2b5-08f8-4410-a2a4-79aaeb855d6e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:25:26.896874 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-d70fe2b5-08f8-4410-a2a4-79aaeb855d6e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:25:26.902938 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1307/2639] 10.222.0.26 () {58 vars in 1232 bytes} [Thu Jul 14 09:25:26 2022] DELETE /identity/v3/projects/39418a3b1c424ed48cfc68d5ac048413 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:25:26.904700 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050b38ff-6306-4be2-963f-4d0235f0d4e8 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:26.905008 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050b38ff-6306-4be2-963f-4d0235f0d4e8 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:26.905287 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050b38ff-6306-4be2-963f-4d0235f0d4e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:26.925764 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-050b38ff-6306-4be2-963f-4d0235f0d4e8 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:25:26.926928 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-050b38ff-6306-4be2-963f-4d0235f0d4e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:25:26.993390 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1333/2640] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:25:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:25:27.001520 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f852a6-44f5-47f3-a8df-835359a5cafa None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:27.001809 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f852a6-44f5-47f3-a8df-835359a5cafa None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:27.002084 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f852a6-44f5-47f3-a8df-835359a5cafa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:27.022076 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-46f852a6-44f5-47f3-a8df-835359a5cafa None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:25:27.023439 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-46f852a6-44f5-47f3-a8df-835359a5cafa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:25:27.032855 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1308/2641] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:25:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:25:27.037196 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050b38ff-6306-4be2-963f-4d0235f0d4e8 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:25:27.047142 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050b38ff-6306-4be2-963f-4d0235f0d4e8 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:25:27.053533 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405399ad-8664-467f-8b5e-47957246276a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:25:27.054205 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405399ad-8664-467f-8b5e-47957246276a admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:27.058525 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405399ad-8664-467f-8b5e-47957246276a admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:27.058525 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405399ad-8664-467f-8b5e-47957246276a admin admin] PATH_INFO: `/v3/domains` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:27.058525 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405399ad-8664-467f-8b5e-47957246276a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:25:27.058525 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405399ad-8664-467f-8b5e-47957246276a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:25:27.058525 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405399ad-8664-467f-8b5e-47957246276a admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:25:27.075508 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1334/2642] 10.222.0.26 () {58 vars in 1152 bytes} [Thu Jul 14 09:25:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:25:27.076916 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f852a6-44f5-47f3-a8df-835359a5cafa None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:25:27.086976 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f852a6-44f5-47f3-a8df-835359a5cafa None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:25:27.092367 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abe0d10-2e73-42ae-8b4a-5cf43948c62f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:25:27.093056 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abe0d10-2e73-42ae-8b4a-5cf43948c62f admin admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:27.093340 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abe0d10-2e73-42ae-8b4a-5cf43948c62f admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:27.093615 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abe0d10-2e73-42ae-8b4a-5cf43948c62f admin admin] PATH_INFO: `/v3/projects` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:27.094373 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abe0d10-2e73-42ae-8b4a-5cf43948c62f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:25:27.096341 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abe0d10-2e73-42ae-8b4a-5cf43948c62f admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:25:27.128102 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1309/2643] 10.222.0.26 () {60 vars in 1152 bytes} [Thu Jul 14 09:25:27 2022] POST /identity/v3/projects => generated 367 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 09:25:27.132472 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405399ad-8664-467f-8b5e-47957246276a admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:25:27.143715 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405399ad-8664-467f-8b5e-47957246276a admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:25:27.176161 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:25:27.208858 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb14f7b-03fa-4d71-ae02-e69c22032dd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:25:27.209534 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb14f7b-03fa-4d71-ae02-e69c22032dd0 admin admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:27.209814 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb14f7b-03fa-4d71-ae02-e69c22032dd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:27.210086 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb14f7b-03fa-4d71-ae02-e69c22032dd0 admin admin] PATH_INFO: `/v3/users` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:27.210793 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb14f7b-03fa-4d71-ae02-e69c22032dd0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:25:27.212744 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb14f7b-03fa-4d71-ae02-e69c22032dd0 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:25:27.234721 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1335/2644] 10.222.0.26 () {60 vars in 1143 bytes} [Thu Jul 14 09:25:27 2022] POST /identity/v3/users => generated 284 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 09:25:27.238913 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abe0d10-2e73-42ae-8b4a-5cf43948c62f admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:25:27.249649 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abe0d10-2e73-42ae-8b4a-5cf43948c62f admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:25:27.255457 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77202d2-e5af-485e-94a4-5f40a2fb555f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:25:27.256123 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77202d2-e5af-485e-94a4-5f40a2fb555f admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:27.256440 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77202d2-e5af-485e-94a4-5f40a2fb555f admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:27.256746 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77202d2-e5af-485e-94a4-5f40a2fb555f admin admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:27.257527 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77202d2-e5af-485e-94a4-5f40a2fb555f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:25:27.259201 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77202d2-e5af-485e-94a4-5f40a2fb555f admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:25:27.274354 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1310/2645] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jul 14 09:25:27 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 09:25:27.276027 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb14f7b-03fa-4d71-ae02-e69c22032dd0 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:25:27.288220 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb14f7b-03fa-4d71-ae02-e69c22032dd0 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:25:27.295837 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b1b832-ef5b-486b-8a47-9114d483cc08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:25:27.296610 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b1b832-ef5b-486b-8a47-9114d483cc08 admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:27.296952 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b1b832-ef5b-486b-8a47-9114d483cc08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:27.297331 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b1b832-ef5b-486b-8a47-9114d483cc08 admin admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:27.298207 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b1b832-ef5b-486b-8a47-9114d483cc08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:25:27.300250 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b1b832-ef5b-486b-8a47-9114d483cc08 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:25:27.312870 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1336/2646] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jul 14 09:25:27 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 09:25:27.316888 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77202d2-e5af-485e-94a4-5f40a2fb555f admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:25:27.330058 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77202d2-e5af-485e-94a4-5f40a2fb555f admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:25:27.337183 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a41c6d8-7e09-4c09-a2f0-1358d0116f87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:25:27.338956 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a41c6d8-7e09-4c09-a2f0-1358d0116f87 admin admin] REQUEST_METHOD: `PUT` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:27.339617 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a41c6d8-7e09-4c09-a2f0-1358d0116f87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:27.340070 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a41c6d8-7e09-4c09-a2f0-1358d0116f87 admin admin] PATH_INFO: `/v3/projects/572ce604bc1a42a9b1e6ee63d08f9c46/users/08acf3434fb246369e5e964c00f22db2/roles/d1691322ce6a4edab7ce722edf0f2988` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:27.360791 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a41c6d8-7e09-4c09-a2f0-1358d0116f87 admin admin] RBAC: Authorizing `identity:create_grant(project_id=572ce604bc1a42a9b1e6ee63d08f9c46, user_id=08acf3434fb246369e5e964c00f22db2, role_id=d1691322ce6a4edab7ce722edf0f2988)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:25:27.363633 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a41c6d8-7e09-4c09-a2f0-1358d0116f87 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:25:27.386928 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1311/2647] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jul 14 09:25:27 2022] PUT /identity/v3/projects/572ce604bc1a42a9b1e6ee63d08f9c46/users/08acf3434fb246369e5e964c00f22db2/roles/d1691322ce6a4edab7ce722edf0f2988 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 09:25:27.390193 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c827dc1-10a7-492d-9203-5322995cb11f None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:27.390597 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c827dc1-10a7-492d-9203-5322995cb11f None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:27.390944 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c827dc1-10a7-492d-9203-5322995cb11f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:27.421561 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-7c827dc1-10a7-492d-9203-5322995cb11f None None] MFA Rules not processed for user `08acf3434fb246369e5e964c00f22db2`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:25:27.422870 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-7c827dc1-10a7-492d-9203-5322995cb11f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:25:27.498089 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1337/2648] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:25:27 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:25:27.502152 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a41c6d8-7e09-4c09-a2f0-1358d0116f87 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:25:27.573623 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a41c6d8-7e09-4c09-a2f0-1358d0116f87 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:25:27.578532 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d7a14d-252b-4196-8938-a03c382a0160 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:25:27.579175 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d7a14d-252b-4196-8938-a03c382a0160 admin admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:27.579501 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d7a14d-252b-4196-8938-a03c382a0160 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:27.579776 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d7a14d-252b-4196-8938-a03c382a0160 admin admin] PATH_INFO: `/v3/projects` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:27.580468 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d7a14d-252b-4196-8938-a03c382a0160 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:25:27.582363 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d7a14d-252b-4196-8938-a03c382a0160 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:25:27.609792 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1312/2649] 10.222.0.26 () {60 vars in 1152 bytes} [Thu Jul 14 09:25:27 2022] POST /identity/v3/projects => generated 369 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 09:25:27.614411 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c827dc1-10a7-492d-9203-5322995cb11f None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:25:27.626130 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c827dc1-10a7-492d-9203-5322995cb11f None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:25:27.692944 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2c7029-808c-463b-aad5-aa5ea9518d64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:25:27.693554 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2c7029-808c-463b-aad5-aa5ea9518d64 admin admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:27.693835 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2c7029-808c-463b-aad5-aa5ea9518d64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:27.694226 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2c7029-808c-463b-aad5-aa5ea9518d64 admin admin] PATH_INFO: `/v3/users` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:27.695123 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2c7029-808c-463b-aad5-aa5ea9518d64 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:25:27.697291 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2c7029-808c-463b-aad5-aa5ea9518d64 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:25:27.724420 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1338/2650] 10.222.0.26 () {60 vars in 1143 bytes} [Thu Jul 14 09:25:27 2022] POST /identity/v3/users => generated 291 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 09:25:27.726293 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d7a14d-252b-4196-8938-a03c382a0160 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:25:27.737331 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d7a14d-252b-4196-8938-a03c382a0160 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:25:27.742215 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b4eb37-6747-4d5c-8286-f71e435a0e59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:25:27.742891 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b4eb37-6747-4d5c-8286-f71e435a0e59 admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:27.743434 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b4eb37-6747-4d5c-8286-f71e435a0e59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:27.743538 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b4eb37-6747-4d5c-8286-f71e435a0e59 admin admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:27.744362 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b4eb37-6747-4d5c-8286-f71e435a0e59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:25:27.745970 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b4eb37-6747-4d5c-8286-f71e435a0e59 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:25:27.761143 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1313/2651] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jul 14 09:25:27 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 09:25:27.762536 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2c7029-808c-463b-aad5-aa5ea9518d64 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:25:27.775213 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2c7029-808c-463b-aad5-aa5ea9518d64 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:25:27.779986 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a891559-2d3f-489f-a6b6-cf6574141175 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:25:27.780892 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a891559-2d3f-489f-a6b6-cf6574141175 admin admin] REQUEST_METHOD: `PUT` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:27.781172 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a891559-2d3f-489f-a6b6-cf6574141175 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:27.781439 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a891559-2d3f-489f-a6b6-cf6574141175 admin admin] PATH_INFO: `/v3/projects/731f10208a224468a9a3297c7c01a996/users/5e7e866d299742ada5474a4b4c9ee64a/roles/7b549d58d1724395b343609d895c80d0` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:27.804627 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a891559-2d3f-489f-a6b6-cf6574141175 admin admin] RBAC: Authorizing `identity:create_grant(project_id=731f10208a224468a9a3297c7c01a996, user_id=5e7e866d299742ada5474a4b4c9ee64a, role_id=7b549d58d1724395b343609d895c80d0)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:25:27.807319 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a891559-2d3f-489f-a6b6-cf6574141175 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:25:27.826960 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1339/2652] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jul 14 09:25:27 2022] PUT /identity/v3/projects/731f10208a224468a9a3297c7c01a996/users/5e7e866d299742ada5474a4b4c9ee64a/roles/7b549d58d1724395b343609d895c80d0 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:25:27.830978 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b4eb37-6747-4d5c-8286-f71e435a0e59 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:25:27.843687 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b4eb37-6747-4d5c-8286-f71e435a0e59 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:25:27.885275 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:25:27.925890 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82537e69-682c-4f69-8c0e-ad3e4cd71675 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:25:27.926562 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82537e69-682c-4f69-8c0e-ad3e4cd71675 admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:27.926843 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82537e69-682c-4f69-8c0e-ad3e4cd71675 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:27.927116 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82537e69-682c-4f69-8c0e-ad3e4cd71675 admin admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:27.927920 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82537e69-682c-4f69-8c0e-ad3e4cd71675 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:25:27.929536 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82537e69-682c-4f69-8c0e-ad3e4cd71675 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:25:27.940889 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1314/2653] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jul 14 09:25:27 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 09:25:27.945069 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a891559-2d3f-489f-a6b6-cf6574141175 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:25:27.950513 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:25:27.956990 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a891559-2d3f-489f-a6b6-cf6574141175 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:25:27.962787 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed27f3e0-92b7-4bd6-b37f-008570fdf7ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:25:27.963870 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed27f3e0-92b7-4bd6-b37f-008570fdf7ef admin admin] REQUEST_METHOD: `PUT` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:27.964180 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed27f3e0-92b7-4bd6-b37f-008570fdf7ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:27.964481 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed27f3e0-92b7-4bd6-b37f-008570fdf7ef admin admin] PATH_INFO: `/v3/projects/731f10208a224468a9a3297c7c01a996/users/5e7e866d299742ada5474a4b4c9ee64a/roles/7b549d58d1724395b343609d895c80d0` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:27.966678 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed27f3e0-92b7-4bd6-b37f-008570fdf7ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=731f10208a224468a9a3297c7c01a996, user_id=5e7e866d299742ada5474a4b4c9ee64a, role_id=7b549d58d1724395b343609d895c80d0)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:25:27.969500 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed27f3e0-92b7-4bd6-b37f-008570fdf7ef admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:25:27.981305 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1340/2654] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jul 14 09:25:27 2022] PUT /identity/v3/projects/731f10208a224468a9a3297c7c01a996/users/5e7e866d299742ada5474a4b4c9ee64a/roles/7b549d58d1724395b343609d895c80d0 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:25:27.991367 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888e91d1-158e-4e72-946d-7671c85894d2 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:27.991758 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888e91d1-158e-4e72-946d-7671c85894d2 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:27.992218 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888e91d1-158e-4e72-946d-7671c85894d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:28.021894 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-888e91d1-158e-4e72-946d-7671c85894d2 None None] MFA Rules not processed for user `5e7e866d299742ada5474a4b4c9ee64a`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:25:28.023167 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-888e91d1-158e-4e72-946d-7671c85894d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:25:28.104130 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1315/2655] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:25:27 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:25:28.116231 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed27f3e0-92b7-4bd6-b37f-008570fdf7ef admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:25:28.118691 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-ed27f3e0-92b7-4bd6-b37f-008570fdf7ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:25:28.187265 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed27f3e0-92b7-4bd6-b37f-008570fdf7ef admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:25:28.191139 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634fb4b8-198f-4574-8f87-36646be9174d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e76f1952a5b146808844b9f2b7395035', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:25:28.191824 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634fb4b8-198f-4574-8f87-36646be9174d service cinder] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:28.192079 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634fb4b8-198f-4574-8f87-36646be9174d service cinder] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:28.192355 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634fb4b8-198f-4574-8f87-36646be9174d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:28.202616 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634fb4b8-198f-4574-8f87-36646be9174d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:25:28.204062 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634fb4b8-198f-4574-8f87-36646be9174d service cinder] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:25:28.214747 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1341/2656] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:25:28 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:25:28.299307 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888e91d1-158e-4e72-946d-7671c85894d2 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:25:28.306405 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888e91d1-158e-4e72-946d-7671c85894d2 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:25:28.310258 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecae66c-82ec-4b6a-85fe-cb1a6ef10513 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e76f1952a5b146808844b9f2b7395035', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:25:28.310926 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecae66c-82ec-4b6a-85fe-cb1a6ef10513 service cinder] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:28.311219 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecae66c-82ec-4b6a-85fe-cb1a6ef10513 service cinder] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:28.311505 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecae66c-82ec-4b6a-85fe-cb1a6ef10513 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:28.381292 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecae66c-82ec-4b6a-85fe-cb1a6ef10513 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:25:28.382882 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecae66c-82ec-4b6a-85fe-cb1a6ef10513 service cinder] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:25:28.393664 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1316/2657] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:25:28 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:25:31.830718 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634fb4b8-198f-4574-8f87-36646be9174d service cinder] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:25:31.895639 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634fb4b8-198f-4574-8f87-36646be9174d service cinder] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:25:31.900827 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e043b3f6-437b-49a8-a3cf-74bd1caf31a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:25:31.901750 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e043b3f6-437b-49a8-a3cf-74bd1caf31a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:31.902015 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e043b3f6-437b-49a8-a3cf-74bd1caf31a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:31.902327 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e043b3f6-437b-49a8-a3cf-74bd1caf31a2 admin admin] PATH_INFO: `/v3/users/08acf3434fb246369e5e964c00f22db2` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:31.903680 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e043b3f6-437b-49a8-a3cf-74bd1caf31a2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=08acf3434fb246369e5e964c00f22db2)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:25:31.905621 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e043b3f6-437b-49a8-a3cf-74bd1caf31a2 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:25:31.953814 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-e043b3f6-437b-49a8-a3cf-74bd1caf31a2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '08acf3434fb246369e5e964c00f22db2'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:25:31.954462 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-e043b3f6-437b-49a8-a3cf-74bd1caf31a2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '08acf3434fb246369e5e964c00f22db2'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:25:31.966627 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-e043b3f6-437b-49a8-a3cf-74bd1caf31a2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '08acf3434fb246369e5e964c00f22db2'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:25:31.975050 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-e043b3f6-437b-49a8-a3cf-74bd1caf31a2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '08acf3434fb246369e5e964c00f22db2'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:25:31.975766 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-e043b3f6-437b-49a8-a3cf-74bd1caf31a2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '08acf3434fb246369e5e964c00f22db2'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:25:31.988750 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1342/2658] 10.222.0.26 () {58 vars in 1223 bytes} [Thu Jul 14 09:25:31 2022] DELETE /identity/v3/users/08acf3434fb246369e5e964c00f22db2 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:25:31.993353 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbe61cd-592d-4eec-bb44-d6bb7bf4135f None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:31.993814 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbe61cd-592d-4eec-bb44-d6bb7bf4135f None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:31.993912 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbe61cd-592d-4eec-bb44-d6bb7bf4135f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:32.012255 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-1dbe61cd-592d-4eec-bb44-d6bb7bf4135f None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:25:32.013332 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-1dbe61cd-592d-4eec-bb44-d6bb7bf4135f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:25:32.057956 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:25:32.079655 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1317/2659] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:25:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:25:32.089429 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e043b3f6-437b-49a8-a3cf-74bd1caf31a2 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:25:32.091318 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-e043b3f6-437b-49a8-a3cf-74bd1caf31a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:25:32.107131 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:25:32.158536 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e043b3f6-437b-49a8-a3cf-74bd1caf31a2 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:25:32.163411 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922cf777-ae8a-48db-8621-da41066d2090 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '952911c492444ffea4b211a24b564117', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:25:32.164111 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922cf777-ae8a-48db-8621-da41066d2090 service neutron] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:32.164396 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922cf777-ae8a-48db-8621-da41066d2090 service neutron] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:32.164671 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922cf777-ae8a-48db-8621-da41066d2090 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:32.176293 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922cf777-ae8a-48db-8621-da41066d2090 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:25:32.177848 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922cf777-ae8a-48db-8621-da41066d2090 service neutron] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:25:32.188651 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1343/2660] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:25:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:25:32.493490 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbe61cd-592d-4eec-bb44-d6bb7bf4135f None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:25:32.494933 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-1dbe61cd-592d-4eec-bb44-d6bb7bf4135f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:25:32.507033 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbe61cd-592d-4eec-bb44-d6bb7bf4135f None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:25:32.512350 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f747cb-1fcb-4e3c-b79d-fecfe99d7ba7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:25:32.513212 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f747cb-1fcb-4e3c-b79d-fecfe99d7ba7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:32.513491 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f747cb-1fcb-4e3c-b79d-fecfe99d7ba7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:32.513759 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f747cb-1fcb-4e3c-b79d-fecfe99d7ba7 admin admin] PATH_INFO: `/v3/projects/572ce604bc1a42a9b1e6ee63d08f9c46` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:32.515026 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f747cb-1fcb-4e3c-b79d-fecfe99d7ba7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=572ce604bc1a42a9b1e6ee63d08f9c46)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:25:32.516975 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f747cb-1fcb-4e3c-b79d-fecfe99d7ba7 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:25:32.570651 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-01f747cb-1fcb-4e3c-b79d-fecfe99d7ba7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '572ce604bc1a42a9b1e6ee63d08f9c46', 'request_id': 'req-01f747cb-1fcb-4e3c-b79d-fecfe99d7ba7'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:25:32.577931 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-01f747cb-1fcb-4e3c-b79d-fecfe99d7ba7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '572ce604bc1a42a9b1e6ee63d08f9c46', 'request_id': 'req-01f747cb-1fcb-4e3c-b79d-fecfe99d7ba7'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:25:32.583798 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-01f747cb-1fcb-4e3c-b79d-fecfe99d7ba7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '572ce604bc1a42a9b1e6ee63d08f9c46', 'request_id': 'req-01f747cb-1fcb-4e3c-b79d-fecfe99d7ba7'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:25:32.599366 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-01f747cb-1fcb-4e3c-b79d-fecfe99d7ba7 admin admin] The token cache is being invalidate because project 572ce604bc1a42a9b1e6ee63d08f9c46 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80381) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 09:25:32.599662 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-01f747cb-1fcb-4e3c-b79d-fecfe99d7ba7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:25:32.600324 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-01f747cb-1fcb-4e3c-b79d-fecfe99d7ba7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:25:32.601987 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1318/2661] 10.222.0.26 () {58 vars in 1232 bytes} [Thu Jul 14 09:25:32 2022] DELETE /identity/v3/projects/572ce604bc1a42a9b1e6ee63d08f9c46 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:25:32.606464 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922cf777-ae8a-48db-8621-da41066d2090 service neutron] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:25:32.616713 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922cf777-ae8a-48db-8621-da41066d2090 service neutron] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:25:32.682777 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0362d7fd-6e29-41a0-992e-9d2b0a182fd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:25:32.683685 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0362d7fd-6e29-41a0-992e-9d2b0a182fd1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:32.683965 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0362d7fd-6e29-41a0-992e-9d2b0a182fd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:32.684230 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0362d7fd-6e29-41a0-992e-9d2b0a182fd1 admin admin] PATH_INFO: `/v3/users/5e7e866d299742ada5474a4b4c9ee64a` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:32.685412 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0362d7fd-6e29-41a0-992e-9d2b0a182fd1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5e7e866d299742ada5474a4b4c9ee64a)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:25:32.687537 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0362d7fd-6e29-41a0-992e-9d2b0a182fd1 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:25:32.737983 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-0362d7fd-6e29-41a0-992e-9d2b0a182fd1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5e7e866d299742ada5474a4b4c9ee64a'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:25:32.738658 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-0362d7fd-6e29-41a0-992e-9d2b0a182fd1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5e7e866d299742ada5474a4b4c9ee64a'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:25:32.751146 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-0362d7fd-6e29-41a0-992e-9d2b0a182fd1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5e7e866d299742ada5474a4b4c9ee64a'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:25:32.759511 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-0362d7fd-6e29-41a0-992e-9d2b0a182fd1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5e7e866d299742ada5474a4b4c9ee64a'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:25:32.760148 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-0362d7fd-6e29-41a0-992e-9d2b0a182fd1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5e7e866d299742ada5474a4b4c9ee64a'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:25:32.774125 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1344/2662] 10.222.0.26 () {58 vars in 1223 bytes} [Thu Jul 14 09:25:32 2022] DELETE /identity/v3/users/5e7e866d299742ada5474a4b4c9ee64a => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:25:33.068037 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f747cb-1fcb-4e3c-b79d-fecfe99d7ba7 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:25:33.069578 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-01f747cb-1fcb-4e3c-b79d-fecfe99d7ba7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:25:33.086933 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f747cb-1fcb-4e3c-b79d-fecfe99d7ba7 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:25:33.147731 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7dc0ee-e895-4af8-96d8-2121d94a78f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:25:33.148518 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7dc0ee-e895-4af8-96d8-2121d94a78f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:33.148795 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7dc0ee-e895-4af8-96d8-2121d94a78f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:33.149068 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7dc0ee-e895-4af8-96d8-2121d94a78f3 admin admin] PATH_INFO: `/v3/projects/731f10208a224468a9a3297c7c01a996` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:33.150252 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7dc0ee-e895-4af8-96d8-2121d94a78f3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=731f10208a224468a9a3297c7c01a996)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:25:33.152187 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7dc0ee-e895-4af8-96d8-2121d94a78f3 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:25:33.198318 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-ae7dc0ee-e895-4af8-96d8-2121d94a78f3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '731f10208a224468a9a3297c7c01a996', 'request_id': 'req-ae7dc0ee-e895-4af8-96d8-2121d94a78f3'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:25:33.204328 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-ae7dc0ee-e895-4af8-96d8-2121d94a78f3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '731f10208a224468a9a3297c7c01a996', 'request_id': 'req-ae7dc0ee-e895-4af8-96d8-2121d94a78f3'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:25:33.209688 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-ae7dc0ee-e895-4af8-96d8-2121d94a78f3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '731f10208a224468a9a3297c7c01a996', 'request_id': 'req-ae7dc0ee-e895-4af8-96d8-2121d94a78f3'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:25:33.213186 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:25:33.223294 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-ae7dc0ee-e895-4af8-96d8-2121d94a78f3 admin admin] The token cache is being invalidate because project 731f10208a224468a9a3297c7c01a996 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80381) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 09:25:33.223711 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-ae7dc0ee-e895-4af8-96d8-2121d94a78f3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:25:33.224825 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-ae7dc0ee-e895-4af8-96d8-2121d94a78f3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:25:33.226729 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1319/2663] 10.222.0.26 () {58 vars in 1232 bytes} [Thu Jul 14 09:25:33 2022] DELETE /identity/v3/projects/731f10208a224468a9a3297c7c01a996 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:25:33.231584 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10982b4d-f7dd-4685-98a0-3090a8144576 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:33.232106 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10982b4d-f7dd-4685-98a0-3090a8144576 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:33.232106 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10982b4d-f7dd-4685-98a0-3090a8144576 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:33.249965 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-10982b4d-f7dd-4685-98a0-3090a8144576 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:25:33.251157 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-10982b4d-f7dd-4685-98a0-3090a8144576 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:25:33.281865 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:25:33.315580 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1345/2664] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:25:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:25:33.325066 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818d912d-1c9d-4799-a82e-664b22440eac None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:33.325466 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818d912d-1c9d-4799-a82e-664b22440eac None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:33.325839 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818d912d-1c9d-4799-a82e-664b22440eac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:33.347351 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-818d912d-1c9d-4799-a82e-664b22440eac None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:25:33.348661 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-818d912d-1c9d-4799-a82e-664b22440eac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:25:33.358902 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1320/2665] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:25:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:25:33.363026 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10982b4d-f7dd-4685-98a0-3090a8144576 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:25:33.374343 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10982b4d-f7dd-4685-98a0-3090a8144576 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:25:33.379367 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55768a21-6f22-447f-be36-a5307cb2781f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:25:33.380204 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55768a21-6f22-447f-be36-a5307cb2781f admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:33.380586 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55768a21-6f22-447f-be36-a5307cb2781f admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:33.380918 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55768a21-6f22-447f-be36-a5307cb2781f admin admin] PATH_INFO: `/v3/domains` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:33.382216 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55768a21-6f22-447f-be36-a5307cb2781f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:25:33.382216 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55768a21-6f22-447f-be36-a5307cb2781f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:25:33.384012 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55768a21-6f22-447f-be36-a5307cb2781f admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:25:33.397453 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1346/2666] 10.222.0.26 () {58 vars in 1152 bytes} [Thu Jul 14 09:25:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 09:25:33.401649 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818d912d-1c9d-4799-a82e-664b22440eac None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:25:33.410880 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818d912d-1c9d-4799-a82e-664b22440eac None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:25:33.415635 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1086c7bf-f3f9-44ee-8e17-1e4d21c32071 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:25:33.416746 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1086c7bf-f3f9-44ee-8e17-1e4d21c32071 admin admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:33.417245 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1086c7bf-f3f9-44ee-8e17-1e4d21c32071 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:33.417735 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1086c7bf-f3f9-44ee-8e17-1e4d21c32071 admin admin] PATH_INFO: `/v3/projects` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:33.418782 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1086c7bf-f3f9-44ee-8e17-1e4d21c32071 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:25:33.420786 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1086c7bf-f3f9-44ee-8e17-1e4d21c32071 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:25:33.454173 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1321/2667] 10.222.0.26 () {60 vars in 1152 bytes} [Thu Jul 14 09:25:33 2022] POST /identity/v3/projects => generated 383 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 09:25:33.459133 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55768a21-6f22-447f-be36-a5307cb2781f admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:25:33.470928 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55768a21-6f22-447f-be36-a5307cb2781f admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:25:33.530923 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45144f6c-c1d1-44e0-ae4a-ba87a631fd34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:25:33.531634 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45144f6c-c1d1-44e0-ae4a-ba87a631fd34 admin admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:33.531918 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45144f6c-c1d1-44e0-ae4a-ba87a631fd34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:33.532190 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45144f6c-c1d1-44e0-ae4a-ba87a631fd34 admin admin] PATH_INFO: `/v3/users` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:33.532887 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45144f6c-c1d1-44e0-ae4a-ba87a631fd34 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:25:33.535046 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45144f6c-c1d1-44e0-ae4a-ba87a631fd34 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:25:33.558883 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1347/2668] 10.222.0.26 () {60 vars in 1143 bytes} [Thu Jul 14 09:25:33 2022] POST /identity/v3/users => generated 292 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 09:25:33.564667 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1086c7bf-f3f9-44ee-8e17-1e4d21c32071 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:25:33.577056 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1086c7bf-f3f9-44ee-8e17-1e4d21c32071 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:25:33.581710 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb7a960-3d87-4ec2-80de-ccfc49550258 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:25:33.582378 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb7a960-3d87-4ec2-80de-ccfc49550258 admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:33.582659 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb7a960-3d87-4ec2-80de-ccfc49550258 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:33.582933 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb7a960-3d87-4ec2-80de-ccfc49550258 admin admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:33.583702 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb7a960-3d87-4ec2-80de-ccfc49550258 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:25:33.585301 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb7a960-3d87-4ec2-80de-ccfc49550258 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:25:33.595788 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1322/2669] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jul 14 09:25:33 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 09:25:33.599920 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45144f6c-c1d1-44e0-ae4a-ba87a631fd34 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:25:33.609948 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45144f6c-c1d1-44e0-ae4a-ba87a631fd34 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:25:33.614511 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3eb4c1c-e883-4e53-8479-7a382196b2d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:25:33.615211 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3eb4c1c-e883-4e53-8479-7a382196b2d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:33.615483 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3eb4c1c-e883-4e53-8479-7a382196b2d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:33.615759 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3eb4c1c-e883-4e53-8479-7a382196b2d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:33.616488 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3eb4c1c-e883-4e53-8479-7a382196b2d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:25:33.618069 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3eb4c1c-e883-4e53-8479-7a382196b2d3 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:25:33.627934 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1348/2670] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jul 14 09:25:33 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 09:25:33.631908 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb7a960-3d87-4ec2-80de-ccfc49550258 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:25:33.641827 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb7a960-3d87-4ec2-80de-ccfc49550258 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:25:33.646951 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444b9a22-b4d9-4ccc-906d-4c65837af318 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:25:33.647821 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444b9a22-b4d9-4ccc-906d-4c65837af318 admin admin] REQUEST_METHOD: `PUT` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:33.648127 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444b9a22-b4d9-4ccc-906d-4c65837af318 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:33.648403 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444b9a22-b4d9-4ccc-906d-4c65837af318 admin admin] PATH_INFO: `/v3/projects/81ab5fa3753c4e2796862b01e7a162e1/users/8b78aa8f7a66485a9f72996d686ffa7c/roles/d1691322ce6a4edab7ce722edf0f2988` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:33.664063 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444b9a22-b4d9-4ccc-906d-4c65837af318 admin admin] RBAC: Authorizing `identity:create_grant(project_id=81ab5fa3753c4e2796862b01e7a162e1, user_id=8b78aa8f7a66485a9f72996d686ffa7c, role_id=d1691322ce6a4edab7ce722edf0f2988)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:25:33.666671 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444b9a22-b4d9-4ccc-906d-4c65837af318 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:25:33.681638 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1323/2671] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jul 14 09:25:33 2022] PUT /identity/v3/projects/81ab5fa3753c4e2796862b01e7a162e1/users/8b78aa8f7a66485a9f72996d686ffa7c/roles/d1691322ce6a4edab7ce722edf0f2988 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:25:33.693065 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e35dd2-ec50-4240-9565-46f606dc1690 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:33.693353 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e35dd2-ec50-4240-9565-46f606dc1690 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:33.693627 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e35dd2-ec50-4240-9565-46f606dc1690 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:33.720501 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-78e35dd2-ec50-4240-9565-46f606dc1690 None None] MFA Rules not processed for user `8b78aa8f7a66485a9f72996d686ffa7c`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:25:33.721703 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-78e35dd2-ec50-4240-9565-46f606dc1690 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:25:33.793512 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1349/2672] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:25:33 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:25:33.798463 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444b9a22-b4d9-4ccc-906d-4c65837af318 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:25:33.872469 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444b9a22-b4d9-4ccc-906d-4c65837af318 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:25:33.877471 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c0b3bd-4865-4ab4-a309-f059252e04e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:25:33.878127 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c0b3bd-4865-4ab4-a309-f059252e04e9 admin admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:33.878411 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c0b3bd-4865-4ab4-a309-f059252e04e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:33.878698 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c0b3bd-4865-4ab4-a309-f059252e04e9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:33.879419 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c0b3bd-4865-4ab4-a309-f059252e04e9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:25:33.881332 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c0b3bd-4865-4ab4-a309-f059252e04e9 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:25:33.920559 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1324/2673] 10.222.0.26 () {60 vars in 1152 bytes} [Thu Jul 14 09:25:33 2022] POST /identity/v3/projects => generated 387 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 09:25:33.921992 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e35dd2-ec50-4240-9565-46f606dc1690 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:25:33.933176 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e35dd2-ec50-4240-9565-46f606dc1690 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:25:33.961327 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:25:33.997500 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8513c5-a670-46fc-a4fd-55004b654499 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:25:33.998438 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8513c5-a670-46fc-a4fd-55004b654499 admin admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:33.998749 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8513c5-a670-46fc-a4fd-55004b654499 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:33.999040 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8513c5-a670-46fc-a4fd-55004b654499 admin admin] PATH_INFO: `/v3/users` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:33.999966 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8513c5-a670-46fc-a4fd-55004b654499 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:25:34.001947 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8513c5-a670-46fc-a4fd-55004b654499 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:25:34.028434 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1350/2674] 10.222.0.26 () {60 vars in 1143 bytes} [Thu Jul 14 09:25:33 2022] POST /identity/v3/users => generated 300 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 09:25:34.029696 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c0b3bd-4865-4ab4-a309-f059252e04e9 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:25:34.040681 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c0b3bd-4865-4ab4-a309-f059252e04e9 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:25:34.045738 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b77843-59e5-44e1-bb0e-30e1cfb9f653 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:25:34.046950 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b77843-59e5-44e1-bb0e-30e1cfb9f653 admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:34.046950 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b77843-59e5-44e1-bb0e-30e1cfb9f653 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:34.047454 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b77843-59e5-44e1-bb0e-30e1cfb9f653 admin admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:34.048122 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b77843-59e5-44e1-bb0e-30e1cfb9f653 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:25:34.049712 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b77843-59e5-44e1-bb0e-30e1cfb9f653 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:25:34.064002 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1325/2675] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jul 14 09:25:34 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 09:25:34.068754 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8513c5-a670-46fc-a4fd-55004b654499 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:25:34.083488 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8513c5-a670-46fc-a4fd-55004b654499 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:25:34.089276 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f132caf-9d4e-40a1-a3ef-1ca1d05017c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:25:34.090236 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f132caf-9d4e-40a1-a3ef-1ca1d05017c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:34.090522 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f132caf-9d4e-40a1-a3ef-1ca1d05017c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:34.090799 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f132caf-9d4e-40a1-a3ef-1ca1d05017c9 admin admin] PATH_INFO: `/v3/projects/4aa6b5198e46432488b39d410d94a79d/users/acc288345c3843be8551bf339ddd4faa/roles/7b549d58d1724395b343609d895c80d0` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:34.110085 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f132caf-9d4e-40a1-a3ef-1ca1d05017c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4aa6b5198e46432488b39d410d94a79d, user_id=acc288345c3843be8551bf339ddd4faa, role_id=7b549d58d1724395b343609d895c80d0)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:25:34.113967 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f132caf-9d4e-40a1-a3ef-1ca1d05017c9 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:25:34.128900 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1351/2676] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jul 14 09:25:34 2022] PUT /identity/v3/projects/4aa6b5198e46432488b39d410d94a79d/users/acc288345c3843be8551bf339ddd4faa/roles/7b549d58d1724395b343609d895c80d0 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:25:34.135041 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b77843-59e5-44e1-bb0e-30e1cfb9f653 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:25:34.147821 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b77843-59e5-44e1-bb0e-30e1cfb9f653 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:25:34.206916 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed5e763-93c5-49f9-b55f-56ffd9eb93c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:25:34.207566 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed5e763-93c5-49f9-b55f-56ffd9eb93c6 admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:34.207849 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed5e763-93c5-49f9-b55f-56ffd9eb93c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:34.208123 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed5e763-93c5-49f9-b55f-56ffd9eb93c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:34.208855 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed5e763-93c5-49f9-b55f-56ffd9eb93c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:25:34.210471 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed5e763-93c5-49f9-b55f-56ffd9eb93c6 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:25:34.220254 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1326/2677] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jul 14 09:25:34 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 09:25:34.224559 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f132caf-9d4e-40a1-a3ef-1ca1d05017c9 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:25:34.235434 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f132caf-9d4e-40a1-a3ef-1ca1d05017c9 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:25:34.240413 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6c4ac8-b47a-47b3-b734-5c449f0aafc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:25:34.241404 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6c4ac8-b47a-47b3-b734-5c449f0aafc6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:34.241680 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6c4ac8-b47a-47b3-b734-5c449f0aafc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:34.242033 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6c4ac8-b47a-47b3-b734-5c449f0aafc6 admin admin] PATH_INFO: `/v3/projects/4aa6b5198e46432488b39d410d94a79d/users/acc288345c3843be8551bf339ddd4faa/roles/7b549d58d1724395b343609d895c80d0` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:34.244124 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6c4ac8-b47a-47b3-b734-5c449f0aafc6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4aa6b5198e46432488b39d410d94a79d, user_id=acc288345c3843be8551bf339ddd4faa, role_id=7b549d58d1724395b343609d895c80d0)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:25:34.246765 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6c4ac8-b47a-47b3-b734-5c449f0aafc6 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:25:34.257474 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1352/2678] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jul 14 09:25:34 2022] PUT /identity/v3/projects/4aa6b5198e46432488b39d410d94a79d/users/acc288345c3843be8551bf339ddd4faa/roles/7b549d58d1724395b343609d895c80d0 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:25:34.268678 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de91b5b7-20c6-4269-866f-db9b39859028 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:34.269052 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de91b5b7-20c6-4269-866f-db9b39859028 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:34.269442 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de91b5b7-20c6-4269-866f-db9b39859028 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:34.278734 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:25:34.303424 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-de91b5b7-20c6-4269-866f-db9b39859028 None None] MFA Rules not processed for user `acc288345c3843be8551bf339ddd4faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:25:34.304655 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-de91b5b7-20c6-4269-866f-db9b39859028 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:25:34.373574 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1327/2679] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:25:34 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:25:34.384956 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6c4ac8-b47a-47b3-b734-5c449f0aafc6 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:25:34.386931 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-ea6c4ac8-b47a-47b3-b734-5c449f0aafc6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:25:34.452662 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6c4ac8-b47a-47b3-b734-5c449f0aafc6 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:25:34.456749 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46990cb-42e0-48e5-b149-1ac61acedfe5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e76f1952a5b146808844b9f2b7395035', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:25:34.457401 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46990cb-42e0-48e5-b149-1ac61acedfe5 service cinder] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:34.457626 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46990cb-42e0-48e5-b149-1ac61acedfe5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:34.457898 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46990cb-42e0-48e5-b149-1ac61acedfe5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:34.472931 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46990cb-42e0-48e5-b149-1ac61acedfe5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:25:34.474845 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46990cb-42e0-48e5-b149-1ac61acedfe5 service cinder] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:25:34.487322 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1353/2680] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:25:34 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:25:34.517330 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de91b5b7-20c6-4269-866f-db9b39859028 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:25:34.589575 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de91b5b7-20c6-4269-866f-db9b39859028 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:25:34.595810 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ffe559-e225-4ea7-ab1d-0c43aa71fe5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:25:34.596706 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ffe559-e225-4ea7-ab1d-0c43aa71fe5e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:34.597021 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ffe559-e225-4ea7-ab1d-0c43aa71fe5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:34.597299 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ffe559-e225-4ea7-ab1d-0c43aa71fe5e admin admin] PATH_INFO: `/v3/users/8b78aa8f7a66485a9f72996d686ffa7c` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:34.598633 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ffe559-e225-4ea7-ab1d-0c43aa71fe5e admin admin] RBAC: Authorizing `identity:delete_user(user_id=8b78aa8f7a66485a9f72996d686ffa7c)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:25:34.601603 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ffe559-e225-4ea7-ab1d-0c43aa71fe5e admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:25:34.657721 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-45ffe559-e225-4ea7-ab1d-0c43aa71fe5e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8b78aa8f7a66485a9f72996d686ffa7c'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:25:34.665457 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-45ffe559-e225-4ea7-ab1d-0c43aa71fe5e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8b78aa8f7a66485a9f72996d686ffa7c'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:25:34.666218 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-45ffe559-e225-4ea7-ab1d-0c43aa71fe5e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8b78aa8f7a66485a9f72996d686ffa7c'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:25:34.666814 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-45ffe559-e225-4ea7-ab1d-0c43aa71fe5e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8b78aa8f7a66485a9f72996d686ffa7c'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:25:34.673664 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-45ffe559-e225-4ea7-ab1d-0c43aa71fe5e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8b78aa8f7a66485a9f72996d686ffa7c'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:25:34.691867 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1328/2681] 10.222.0.26 () {58 vars in 1223 bytes} [Thu Jul 14 09:25:34 2022] DELETE /identity/v3/users/8b78aa8f7a66485a9f72996d686ffa7c => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:25:34.696529 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e108362-1c8a-48dc-afb3-849c2d5748b8 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:34.696817 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e108362-1c8a-48dc-afb3-849c2d5748b8 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:34.697092 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e108362-1c8a-48dc-afb3-849c2d5748b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:34.716071 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-2e108362-1c8a-48dc-afb3-849c2d5748b8 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:25:34.717205 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-2e108362-1c8a-48dc-afb3-849c2d5748b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:25:34.780878 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1354/2682] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:25:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:25:34.790916 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ffe559-e225-4ea7-ab1d-0c43aa71fe5e admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:25:34.792580 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-45ffe559-e225-4ea7-ab1d-0c43aa71fe5e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:25:34.803516 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:25:34.854721 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ffe559-e225-4ea7-ab1d-0c43aa71fe5e admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:25:34.859088 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaeeb06d-76aa-4e2b-9ee3-2b387448be05 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '952911c492444ffea4b211a24b564117', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:25:34.859755 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaeeb06d-76aa-4e2b-9ee3-2b387448be05 service neutron] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:34.860042 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaeeb06d-76aa-4e2b-9ee3-2b387448be05 service neutron] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:34.860320 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaeeb06d-76aa-4e2b-9ee3-2b387448be05 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:34.870169 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaeeb06d-76aa-4e2b-9ee3-2b387448be05 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:25:34.871668 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaeeb06d-76aa-4e2b-9ee3-2b387448be05 service neutron] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:25:34.882018 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1329/2683] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:25:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:25:35.167334 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e108362-1c8a-48dc-afb3-849c2d5748b8 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:25:35.168886 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-2e108362-1c8a-48dc-afb3-849c2d5748b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:25:35.181469 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e108362-1c8a-48dc-afb3-849c2d5748b8 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:25:35.189107 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e97477-f78b-4a85-a811-81b72a826bd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:25:35.189282 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e97477-f78b-4a85-a811-81b72a826bd3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:35.189497 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e97477-f78b-4a85-a811-81b72a826bd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:35.189786 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e97477-f78b-4a85-a811-81b72a826bd3 admin admin] PATH_INFO: `/v3/projects/81ab5fa3753c4e2796862b01e7a162e1` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:35.191017 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e97477-f78b-4a85-a811-81b72a826bd3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=81ab5fa3753c4e2796862b01e7a162e1)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:25:35.193017 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e97477-f78b-4a85-a811-81b72a826bd3 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:25:35.215144 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:25:35.243919 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-73e97477-f78b-4a85-a811-81b72a826bd3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '81ab5fa3753c4e2796862b01e7a162e1', 'request_id': 'req-73e97477-f78b-4a85-a811-81b72a826bd3'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:25:35.250059 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-73e97477-f78b-4a85-a811-81b72a826bd3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '81ab5fa3753c4e2796862b01e7a162e1', 'request_id': 'req-73e97477-f78b-4a85-a811-81b72a826bd3'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:25:35.255624 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-73e97477-f78b-4a85-a811-81b72a826bd3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '81ab5fa3753c4e2796862b01e7a162e1', 'request_id': 'req-73e97477-f78b-4a85-a811-81b72a826bd3'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:25:35.269047 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-73e97477-f78b-4a85-a811-81b72a826bd3 admin admin] The token cache is being invalidate because project 81ab5fa3753c4e2796862b01e7a162e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80382) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 09:25:35.269336 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-73e97477-f78b-4a85-a811-81b72a826bd3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:25:35.270277 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-73e97477-f78b-4a85-a811-81b72a826bd3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:25:35.271888 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1355/2684] 10.222.0.26 () {58 vars in 1232 bytes} [Thu Jul 14 09:25:35 2022] DELETE /identity/v3/projects/81ab5fa3753c4e2796862b01e7a162e1 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:25:35.275861 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaeeb06d-76aa-4e2b-9ee3-2b387448be05 service neutron] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:25:35.285775 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaeeb06d-76aa-4e2b-9ee3-2b387448be05 service neutron] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:25:35.345125 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393dc5fe-b22c-45c5-a4f2-c188a045d6c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:25:35.346038 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393dc5fe-b22c-45c5-a4f2-c188a045d6c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:35.346323 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393dc5fe-b22c-45c5-a4f2-c188a045d6c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:35.346595 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393dc5fe-b22c-45c5-a4f2-c188a045d6c9 admin admin] PATH_INFO: `/v3/users/acc288345c3843be8551bf339ddd4faa` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:35.347927 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393dc5fe-b22c-45c5-a4f2-c188a045d6c9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=acc288345c3843be8551bf339ddd4faa)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:25:35.349926 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393dc5fe-b22c-45c5-a4f2-c188a045d6c9 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:25:35.398518 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-393dc5fe-b22c-45c5-a4f2-c188a045d6c9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'acc288345c3843be8551bf339ddd4faa'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:25:35.406631 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-393dc5fe-b22c-45c5-a4f2-c188a045d6c9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'acc288345c3843be8551bf339ddd4faa'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:25:35.407285 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-393dc5fe-b22c-45c5-a4f2-c188a045d6c9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'acc288345c3843be8551bf339ddd4faa'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:25:35.407982 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-393dc5fe-b22c-45c5-a4f2-c188a045d6c9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'acc288345c3843be8551bf339ddd4faa'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:25:35.415201 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-393dc5fe-b22c-45c5-a4f2-c188a045d6c9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'acc288345c3843be8551bf339ddd4faa'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:25:35.432250 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1330/2685] 10.222.0.26 () {58 vars in 1223 bytes} [Thu Jul 14 09:25:35 2022] DELETE /identity/v3/users/acc288345c3843be8551bf339ddd4faa => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:25:35.737275 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e97477-f78b-4a85-a811-81b72a826bd3 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:25:35.739283 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-73e97477-f78b-4a85-a811-81b72a826bd3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:25:35.755777 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e97477-f78b-4a85-a811-81b72a826bd3 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:25:35.823947 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1ac4ee-1935-40c2-9a5b-9a101cd434e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:25:35.824989 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1ac4ee-1935-40c2-9a5b-9a101cd434e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:35.825272 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1ac4ee-1935-40c2-9a5b-9a101cd434e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:35.825547 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1ac4ee-1935-40c2-9a5b-9a101cd434e6 admin admin] PATH_INFO: `/v3/projects/4aa6b5198e46432488b39d410d94a79d` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:35.826784 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1ac4ee-1935-40c2-9a5b-9a101cd434e6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4aa6b5198e46432488b39d410d94a79d)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:25:35.828821 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1ac4ee-1935-40c2-9a5b-9a101cd434e6 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:25:35.876197 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-1f1ac4ee-1935-40c2-9a5b-9a101cd434e6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4aa6b5198e46432488b39d410d94a79d', 'request_id': 'req-1f1ac4ee-1935-40c2-9a5b-9a101cd434e6'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:25:35.882308 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-1f1ac4ee-1935-40c2-9a5b-9a101cd434e6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4aa6b5198e46432488b39d410d94a79d', 'request_id': 'req-1f1ac4ee-1935-40c2-9a5b-9a101cd434e6'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:25:35.887778 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-1f1ac4ee-1935-40c2-9a5b-9a101cd434e6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4aa6b5198e46432488b39d410d94a79d', 'request_id': 'req-1f1ac4ee-1935-40c2-9a5b-9a101cd434e6'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:25:35.901752 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-1f1ac4ee-1935-40c2-9a5b-9a101cd434e6 admin admin] The token cache is being invalidate because project 4aa6b5198e46432488b39d410d94a79d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80382) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 09:25:35.902047 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-1f1ac4ee-1935-40c2-9a5b-9a101cd434e6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:25:35.902666 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-1f1ac4ee-1935-40c2-9a5b-9a101cd434e6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:25:35.904249 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1356/2686] 10.222.0.26 () {58 vars in 1232 bytes} [Thu Jul 14 09:25:35 2022] DELETE /identity/v3/projects/4aa6b5198e46432488b39d410d94a79d => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:25:35.909355 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a37c137-ac43-4dae-976b-b27482055c25 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:35.909643 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a37c137-ac43-4dae-976b-b27482055c25 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:35.909918 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a37c137-ac43-4dae-976b-b27482055c25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:35.928540 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-8a37c137-ac43-4dae-976b-b27482055c25 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:25:35.929800 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-8a37c137-ac43-4dae-976b-b27482055c25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:25:35.955369 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:25:35.993741 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1331/2687] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:25:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:25:36.001929 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315ccaa0-f616-448a-a5e7-53f9b2b6d11a None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:36.002216 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315ccaa0-f616-448a-a5e7-53f9b2b6d11a None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:36.002498 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315ccaa0-f616-448a-a5e7-53f9b2b6d11a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:36.022128 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-315ccaa0-f616-448a-a5e7-53f9b2b6d11a None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:25:36.023338 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-315ccaa0-f616-448a-a5e7-53f9b2b6d11a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:25:36.031836 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1357/2688] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:25:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 09:25:36.035929 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a37c137-ac43-4dae-976b-b27482055c25 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:25:36.045871 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a37c137-ac43-4dae-976b-b27482055c25 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:25:36.050928 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3e70e1-5eef-487b-96a1-b15b65231a06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:25:36.051620 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3e70e1-5eef-487b-96a1-b15b65231a06 admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:36.051908 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3e70e1-5eef-487b-96a1-b15b65231a06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:36.052159 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3e70e1-5eef-487b-96a1-b15b65231a06 admin admin] PATH_INFO: `/v3/domains` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:36.052910 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3e70e1-5eef-487b-96a1-b15b65231a06 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:25:36.053217 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3e70e1-5eef-487b-96a1-b15b65231a06 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:25:36.054826 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3e70e1-5eef-487b-96a1-b15b65231a06 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:25:36.066919 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1332/2689] 10.222.0.26 () {58 vars in 1152 bytes} [Thu Jul 14 09:25:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:25:36.071226 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315ccaa0-f616-448a-a5e7-53f9b2b6d11a None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:25:36.081015 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315ccaa0-f616-448a-a5e7-53f9b2b6d11a None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:25:36.086618 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98258f14-479d-4a97-8653-d37e0c404335 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:25:36.087322 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98258f14-479d-4a97-8653-d37e0c404335 admin admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:36.087617 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98258f14-479d-4a97-8653-d37e0c404335 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:36.087891 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98258f14-479d-4a97-8653-d37e0c404335 admin admin] PATH_INFO: `/v3/projects` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:36.088581 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98258f14-479d-4a97-8653-d37e0c404335 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:25:36.090533 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98258f14-479d-4a97-8653-d37e0c404335 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:25:36.118548 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1358/2690] 10.222.0.26 () {60 vars in 1152 bytes} [Thu Jul 14 09:25:36 2022] POST /identity/v3/projects => generated 407 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 09:25:36.124767 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3e70e1-5eef-487b-96a1-b15b65231a06 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:25:36.135699 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3e70e1-5eef-487b-96a1-b15b65231a06 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:25:36.192197 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afbe0af-b19e-49c5-841d-d6659076fd31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:25:36.192867 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afbe0af-b19e-49c5-841d-d6659076fd31 admin admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:36.193163 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afbe0af-b19e-49c5-841d-d6659076fd31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:36.193440 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afbe0af-b19e-49c5-841d-d6659076fd31 admin admin] PATH_INFO: `/v3/users` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:36.194159 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afbe0af-b19e-49c5-841d-d6659076fd31 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:25:36.196217 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afbe0af-b19e-49c5-841d-d6659076fd31 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:25:36.219007 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1333/2691] 10.222.0.26 () {60 vars in 1143 bytes} [Thu Jul 14 09:25:36 2022] POST /identity/v3/users => generated 304 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 09:25:36.223353 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98258f14-479d-4a97-8653-d37e0c404335 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:25:36.234456 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98258f14-479d-4a97-8653-d37e0c404335 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:25:36.240109 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69792e25-b9d0-4977-a363-1b5f976301f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:25:36.240899 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69792e25-b9d0-4977-a363-1b5f976301f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:36.241402 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69792e25-b9d0-4977-a363-1b5f976301f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:36.241775 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69792e25-b9d0-4977-a363-1b5f976301f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:36.242730 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69792e25-b9d0-4977-a363-1b5f976301f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:25:36.244404 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69792e25-b9d0-4977-a363-1b5f976301f0 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:25:36.246901 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:25:36.259555 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1359/2692] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jul 14 09:25:36 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 09:25:36.261313 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afbe0af-b19e-49c5-841d-d6659076fd31 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:25:36.272572 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afbe0af-b19e-49c5-841d-d6659076fd31 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:25:36.278106 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc56fbc6-22f3-441f-9128-02ad75dc78e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:25:36.278797 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc56fbc6-22f3-441f-9128-02ad75dc78e2 admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:36.279088 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc56fbc6-22f3-441f-9128-02ad75dc78e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:36.279420 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc56fbc6-22f3-441f-9128-02ad75dc78e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:36.280174 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc56fbc6-22f3-441f-9128-02ad75dc78e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:25:36.282031 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc56fbc6-22f3-441f-9128-02ad75dc78e2 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:25:36.295453 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1334/2693] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jul 14 09:25:36 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 09:25:36.298543 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69792e25-b9d0-4977-a363-1b5f976301f0 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:25:36.309023 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69792e25-b9d0-4977-a363-1b5f976301f0 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:25:36.314041 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e05a4fb-efa1-4643-9ab1-33977bd20b5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:25:36.314923 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e05a4fb-efa1-4643-9ab1-33977bd20b5a admin admin] REQUEST_METHOD: `PUT` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:36.315216 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e05a4fb-efa1-4643-9ab1-33977bd20b5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:36.315494 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e05a4fb-efa1-4643-9ab1-33977bd20b5a admin admin] PATH_INFO: `/v3/projects/de9f3a5158164f1abc0b6090e83e783c/users/8c43c7d53825406188e26ecb1e5dc15a/roles/d1691322ce6a4edab7ce722edf0f2988` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:36.329930 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e05a4fb-efa1-4643-9ab1-33977bd20b5a admin admin] RBAC: Authorizing `identity:create_grant(project_id=de9f3a5158164f1abc0b6090e83e783c, user_id=8c43c7d53825406188e26ecb1e5dc15a, role_id=d1691322ce6a4edab7ce722edf0f2988)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:25:36.332590 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e05a4fb-efa1-4643-9ab1-33977bd20b5a admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:25:36.347151 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1360/2694] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jul 14 09:25:36 2022] PUT /identity/v3/projects/de9f3a5158164f1abc0b6090e83e783c/users/8c43c7d53825406188e26ecb1e5dc15a/roles/d1691322ce6a4edab7ce722edf0f2988 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:25:36.358498 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e39e0e-d962-475b-85cc-fb94191d3bb3 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:36.358790 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e39e0e-d962-475b-85cc-fb94191d3bb3 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:36.359066 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e39e0e-d962-475b-85cc-fb94191d3bb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:36.390798 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-77e39e0e-d962-475b-85cc-fb94191d3bb3 None None] MFA Rules not processed for user `8c43c7d53825406188e26ecb1e5dc15a`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:25:36.392169 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-77e39e0e-d962-475b-85cc-fb94191d3bb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:25:36.460786 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1335/2695] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:25:36 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:25:36.465739 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e05a4fb-efa1-4643-9ab1-33977bd20b5a admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:25:36.532394 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e05a4fb-efa1-4643-9ab1-33977bd20b5a admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:25:36.537441 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98f2998-7e16-499e-a38d-474a66e05ae0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:25:36.538121 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98f2998-7e16-499e-a38d-474a66e05ae0 admin admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:36.538408 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98f2998-7e16-499e-a38d-474a66e05ae0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:36.538692 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98f2998-7e16-499e-a38d-474a66e05ae0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:36.539417 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98f2998-7e16-499e-a38d-474a66e05ae0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:25:36.541367 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98f2998-7e16-499e-a38d-474a66e05ae0 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:25:36.573100 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1361/2696] 10.222.0.26 () {60 vars in 1152 bytes} [Thu Jul 14 09:25:36 2022] POST /identity/v3/projects => generated 409 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 09:25:36.577694 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e39e0e-d962-475b-85cc-fb94191d3bb3 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:25:36.589723 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e39e0e-d962-475b-85cc-fb94191d3bb3 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:25:36.610339 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:25:36.648478 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3921010-b03c-4a9f-99dd-3f4d3f082e7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:25:36.649146 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3921010-b03c-4a9f-99dd-3f4d3f082e7b admin admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:36.649424 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3921010-b03c-4a9f-99dd-3f4d3f082e7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:36.649698 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3921010-b03c-4a9f-99dd-3f4d3f082e7b admin admin] PATH_INFO: `/v3/users` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:36.650403 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3921010-b03c-4a9f-99dd-3f4d3f082e7b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:25:36.652391 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3921010-b03c-4a9f-99dd-3f4d3f082e7b admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:25:36.675517 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1336/2697] 10.222.0.26 () {60 vars in 1143 bytes} [Thu Jul 14 09:25:36 2022] POST /identity/v3/users => generated 311 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 09:25:36.679782 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98f2998-7e16-499e-a38d-474a66e05ae0 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:25:36.691182 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98f2998-7e16-499e-a38d-474a66e05ae0 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:25:36.696441 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabe0585-078c-4404-b0ec-a3426ea0a44a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:25:36.697278 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabe0585-078c-4404-b0ec-a3426ea0a44a admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:36.697556 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabe0585-078c-4404-b0ec-a3426ea0a44a admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:36.697834 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabe0585-078c-4404-b0ec-a3426ea0a44a admin admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:36.698599 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabe0585-078c-4404-b0ec-a3426ea0a44a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:25:36.700300 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabe0585-078c-4404-b0ec-a3426ea0a44a admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:25:36.711054 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1362/2698] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jul 14 09:25:36 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 09:25:36.715700 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3921010-b03c-4a9f-99dd-3f4d3f082e7b admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:25:36.726511 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3921010-b03c-4a9f-99dd-3f4d3f082e7b admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:25:36.732441 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3198c0b3-717a-47e6-b91d-0459c66b6be7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:25:36.733386 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3198c0b3-717a-47e6-b91d-0459c66b6be7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:36.733673 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3198c0b3-717a-47e6-b91d-0459c66b6be7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:36.733947 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3198c0b3-717a-47e6-b91d-0459c66b6be7 admin admin] PATH_INFO: `/v3/projects/f9491a680e4941428a0cd9bc6b1e4e40/users/12333f1566524405a38401efb77c124f/roles/7b549d58d1724395b343609d895c80d0` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:36.750337 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3198c0b3-717a-47e6-b91d-0459c66b6be7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9491a680e4941428a0cd9bc6b1e4e40, user_id=12333f1566524405a38401efb77c124f, role_id=7b549d58d1724395b343609d895c80d0)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:25:36.753063 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3198c0b3-717a-47e6-b91d-0459c66b6be7 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:25:36.769321 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1337/2699] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jul 14 09:25:36 2022] PUT /identity/v3/projects/f9491a680e4941428a0cd9bc6b1e4e40/users/12333f1566524405a38401efb77c124f/roles/7b549d58d1724395b343609d895c80d0 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:25:36.774657 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabe0585-078c-4404-b0ec-a3426ea0a44a admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:25:36.786664 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabe0585-078c-4404-b0ec-a3426ea0a44a admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:25:36.844537 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d3a9e9-eaf4-4ea4-afe5-2462a885353f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:25:36.845219 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d3a9e9-eaf4-4ea4-afe5-2462a885353f admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:36.845502 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d3a9e9-eaf4-4ea4-afe5-2462a885353f admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:36.845776 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d3a9e9-eaf4-4ea4-afe5-2462a885353f admin admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:36.846540 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d3a9e9-eaf4-4ea4-afe5-2462a885353f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:25:36.848390 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d3a9e9-eaf4-4ea4-afe5-2462a885353f admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:25:36.863179 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1363/2700] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jul 14 09:25:36 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 09:25:36.864501 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3198c0b3-717a-47e6-b91d-0459c66b6be7 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:25:36.879412 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3198c0b3-717a-47e6-b91d-0459c66b6be7 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:25:36.882326 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6192ea70-5f02-479e-8a1b-4a39c7c839c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:25:36.883498 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6192ea70-5f02-479e-8a1b-4a39c7c839c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:36.883973 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6192ea70-5f02-479e-8a1b-4a39c7c839c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:36.884246 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6192ea70-5f02-479e-8a1b-4a39c7c839c5 admin admin] PATH_INFO: `/v3/projects/f9491a680e4941428a0cd9bc6b1e4e40/users/12333f1566524405a38401efb77c124f/roles/7b549d58d1724395b343609d895c80d0` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:36.886215 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6192ea70-5f02-479e-8a1b-4a39c7c839c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9491a680e4941428a0cd9bc6b1e4e40, user_id=12333f1566524405a38401efb77c124f, role_id=7b549d58d1724395b343609d895c80d0)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:25:36.888967 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6192ea70-5f02-479e-8a1b-4a39c7c839c5 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:25:36.900068 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1338/2701] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jul 14 09:25:36 2022] PUT /identity/v3/projects/f9491a680e4941428a0cd9bc6b1e4e40/users/12333f1566524405a38401efb77c124f/roles/7b549d58d1724395b343609d895c80d0 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 09:25:36.910363 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b6e2b2-3b33-492c-bf4b-861f0feffc29 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:36.910652 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b6e2b2-3b33-492c-bf4b-861f0feffc29 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:36.910932 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b6e2b2-3b33-492c-bf4b-861f0feffc29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:36.937320 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-d6b6e2b2-3b33-492c-bf4b-861f0feffc29 None None] MFA Rules not processed for user `12333f1566524405a38401efb77c124f`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:25:36.938541 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-d6b6e2b2-3b33-492c-bf4b-861f0feffc29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:25:36.972500 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:25:37.017518 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1364/2702] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:25:36 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 09:25:37.028915 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6192ea70-5f02-479e-8a1b-4a39c7c839c5 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:25:37.030708 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-6192ea70-5f02-479e-8a1b-4a39c7c839c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:25:37.095413 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6192ea70-5f02-479e-8a1b-4a39c7c839c5 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:25:37.101845 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4027ac01-b966-4215-baa3-9ab7d63c628d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e76f1952a5b146808844b9f2b7395035', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:25:37.102750 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4027ac01-b966-4215-baa3-9ab7d63c628d service cinder] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:37.103122 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4027ac01-b966-4215-baa3-9ab7d63c628d service cinder] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:37.103534 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4027ac01-b966-4215-baa3-9ab7d63c628d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:37.114802 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4027ac01-b966-4215-baa3-9ab7d63c628d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:25:37.116247 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4027ac01-b966-4215-baa3-9ab7d63c628d service cinder] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:25:37.126087 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1339/2703] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:25:37 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:25:37.296740 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b6e2b2-3b33-492c-bf4b-861f0feffc29 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:25:37.298300 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-d6b6e2b2-3b33-492c-bf4b-861f0feffc29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:25:37.310574 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b6e2b2-3b33-492c-bf4b-861f0feffc29 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:25:37.314656 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67058622-4e48-4908-9f3b-373ceea1dade service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e76f1952a5b146808844b9f2b7395035', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:25:37.315473 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67058622-4e48-4908-9f3b-373ceea1dade service cinder] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:37.315818 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67058622-4e48-4908-9f3b-373ceea1dade service cinder] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:37.316157 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67058622-4e48-4908-9f3b-373ceea1dade service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:37.372832 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67058622-4e48-4908-9f3b-373ceea1dade service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:25:37.374176 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67058622-4e48-4908-9f3b-373ceea1dade service cinder] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:25:37.386691 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1365/2704] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:25:37 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:25:40.223666 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4027ac01-b966-4215-baa3-9ab7d63c628d service cinder] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:25:40.225139 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-4027ac01-b966-4215-baa3-9ab7d63c628d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:25:40.293115 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4027ac01-b966-4215-baa3-9ab7d63c628d service cinder] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:25:40.298283 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574b6d3d-f302-4934-a674-b25cec6301c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:25:40.299237 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574b6d3d-f302-4934-a674-b25cec6301c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:40.299560 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574b6d3d-f302-4934-a674-b25cec6301c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:40.299881 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574b6d3d-f302-4934-a674-b25cec6301c7 admin admin] PATH_INFO: `/v3/users/ba01a495640d4ab2803bbc1872e94dbc` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:40.301225 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574b6d3d-f302-4934-a674-b25cec6301c7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ba01a495640d4ab2803bbc1872e94dbc)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:25:40.303777 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574b6d3d-f302-4934-a674-b25cec6301c7 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:25:40.326263 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:25:40.356158 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-574b6d3d-f302-4934-a674-b25cec6301c7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ba01a495640d4ab2803bbc1872e94dbc'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:25:40.364615 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-574b6d3d-f302-4934-a674-b25cec6301c7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ba01a495640d4ab2803bbc1872e94dbc'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:25:40.365260 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-574b6d3d-f302-4934-a674-b25cec6301c7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ba01a495640d4ab2803bbc1872e94dbc'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:25:40.365900 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-574b6d3d-f302-4934-a674-b25cec6301c7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ba01a495640d4ab2803bbc1872e94dbc'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:25:40.373284 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-574b6d3d-f302-4934-a674-b25cec6301c7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ba01a495640d4ab2803bbc1872e94dbc'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:25:40.396723 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1340/2705] 10.222.0.26 () {58 vars in 1223 bytes} [Thu Jul 14 09:25:40 2022] DELETE /identity/v3/users/ba01a495640d4ab2803bbc1872e94dbc => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:25:40.401336 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722affba-512a-46c4-8cc0-0f13877a1140 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:40.401825 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722affba-512a-46c4-8cc0-0f13877a1140 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:40.401949 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722affba-512a-46c4-8cc0-0f13877a1140 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:40.423838 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-722affba-512a-46c4-8cc0-0f13877a1140 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:25:40.425147 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-722affba-512a-46c4-8cc0-0f13877a1140 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:25:40.501320 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1366/2706] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:25:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:25:40.511788 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574b6d3d-f302-4934-a674-b25cec6301c7 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:25:40.513195 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-574b6d3d-f302-4934-a674-b25cec6301c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:25:40.598783 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574b6d3d-f302-4934-a674-b25cec6301c7 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:25:40.602334 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d11c3d3-0b85-4497-a48d-3f14af0db33e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '952911c492444ffea4b211a24b564117', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:25:40.602977 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d11c3d3-0b85-4497-a48d-3f14af0db33e service neutron] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:40.603285 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d11c3d3-0b85-4497-a48d-3f14af0db33e service neutron] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:40.603558 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d11c3d3-0b85-4497-a48d-3f14af0db33e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:40.635166 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d11c3d3-0b85-4497-a48d-3f14af0db33e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:25:40.636755 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d11c3d3-0b85-4497-a48d-3f14af0db33e service neutron] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:25:40.669328 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1341/2707] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:25:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:25:40.945888 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722affba-512a-46c4-8cc0-0f13877a1140 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:25:40.947361 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-722affba-512a-46c4-8cc0-0f13877a1140 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:25:41.027082 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722affba-512a-46c4-8cc0-0f13877a1140 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:25:41.031964 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837e74e2-52f8-471c-85c4-dc65373e2d6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:25:41.032834 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837e74e2-52f8-471c-85c4-dc65373e2d6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:41.033113 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837e74e2-52f8-471c-85c4-dc65373e2d6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:41.033383 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837e74e2-52f8-471c-85c4-dc65373e2d6f admin admin] PATH_INFO: `/v3/projects/0bcc0bfc95694837a3feb523b77e6eb3` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:41.034592 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837e74e2-52f8-471c-85c4-dc65373e2d6f admin admin] RBAC: Authorizing `identity:delete_project(project_id=0bcc0bfc95694837a3feb523b77e6eb3)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:25:41.036522 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837e74e2-52f8-471c-85c4-dc65373e2d6f admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:25:41.037976 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:25:41.084354 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-837e74e2-52f8-471c-85c4-dc65373e2d6f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0bcc0bfc95694837a3feb523b77e6eb3', 'request_id': 'req-837e74e2-52f8-471c-85c4-dc65373e2d6f'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:25:41.090508 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-837e74e2-52f8-471c-85c4-dc65373e2d6f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0bcc0bfc95694837a3feb523b77e6eb3', 'request_id': 'req-837e74e2-52f8-471c-85c4-dc65373e2d6f'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:25:41.096821 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-837e74e2-52f8-471c-85c4-dc65373e2d6f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0bcc0bfc95694837a3feb523b77e6eb3', 'request_id': 'req-837e74e2-52f8-471c-85c4-dc65373e2d6f'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:25:41.110260 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-837e74e2-52f8-471c-85c4-dc65373e2d6f admin admin] The token cache is being invalidate because project 0bcc0bfc95694837a3feb523b77e6eb3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80382) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 09:25:41.110549 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-837e74e2-52f8-471c-85c4-dc65373e2d6f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:25:41.111164 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-837e74e2-52f8-471c-85c4-dc65373e2d6f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:25:41.113288 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1367/2708] 10.222.0.26 () {58 vars in 1232 bytes} [Thu Jul 14 09:25:40 2022] DELETE /identity/v3/projects/0bcc0bfc95694837a3feb523b77e6eb3 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:25:41.117494 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d11c3d3-0b85-4497-a48d-3f14af0db33e service neutron] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:25:41.127740 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d11c3d3-0b85-4497-a48d-3f14af0db33e service neutron] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:25:41.178646 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:25:41.190374 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992c8ebe-153c-4d89-aea0-04f652a8e772 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:25:41.191528 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992c8ebe-153c-4d89-aea0-04f652a8e772 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:41.192058 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992c8ebe-153c-4d89-aea0-04f652a8e772 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:41.192350 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992c8ebe-153c-4d89-aea0-04f652a8e772 admin admin] PATH_INFO: `/v3/users/468bb62f19ca45adb7183eef1481071e` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:41.193602 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992c8ebe-153c-4d89-aea0-04f652a8e772 admin admin] RBAC: Authorizing `identity:delete_user(user_id=468bb62f19ca45adb7183eef1481071e)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:25:41.195631 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992c8ebe-153c-4d89-aea0-04f652a8e772 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:25:41.244498 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-992c8ebe-153c-4d89-aea0-04f652a8e772 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '468bb62f19ca45adb7183eef1481071e'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:25:41.253028 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-992c8ebe-153c-4d89-aea0-04f652a8e772 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '468bb62f19ca45adb7183eef1481071e'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:25:41.253757 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-992c8ebe-153c-4d89-aea0-04f652a8e772 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '468bb62f19ca45adb7183eef1481071e'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:25:41.254501 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-992c8ebe-153c-4d89-aea0-04f652a8e772 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '468bb62f19ca45adb7183eef1481071e'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:25:41.262013 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-992c8ebe-153c-4d89-aea0-04f652a8e772 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '468bb62f19ca45adb7183eef1481071e'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:25:41.279816 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1342/2709] 10.222.0.26 () {58 vars in 1223 bytes} [Thu Jul 14 09:25:41 2022] DELETE /identity/v3/users/468bb62f19ca45adb7183eef1481071e => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:25:41.567702 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837e74e2-52f8-471c-85c4-dc65373e2d6f admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:25:41.570732 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-837e74e2-52f8-471c-85c4-dc65373e2d6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:25:41.596983 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837e74e2-52f8-471c-85c4-dc65373e2d6f admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:25:41.661144 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971581b6-cd03-4fb2-bb0a-9adebc49fa89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:25:41.661999 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971581b6-cd03-4fb2-bb0a-9adebc49fa89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:41.662337 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971581b6-cd03-4fb2-bb0a-9adebc49fa89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:41.662596 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971581b6-cd03-4fb2-bb0a-9adebc49fa89 admin admin] PATH_INFO: `/v3/projects/8056e60ccbac44cdb0a2b9bd66e8e745` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:41.663884 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971581b6-cd03-4fb2-bb0a-9adebc49fa89 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8056e60ccbac44cdb0a2b9bd66e8e745)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:25:41.666124 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971581b6-cd03-4fb2-bb0a-9adebc49fa89 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:25:41.716244 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-971581b6-cd03-4fb2-bb0a-9adebc49fa89 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8056e60ccbac44cdb0a2b9bd66e8e745', 'request_id': 'req-971581b6-cd03-4fb2-bb0a-9adebc49fa89'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:25:41.722738 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-971581b6-cd03-4fb2-bb0a-9adebc49fa89 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8056e60ccbac44cdb0a2b9bd66e8e745', 'request_id': 'req-971581b6-cd03-4fb2-bb0a-9adebc49fa89'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:25:41.728510 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-971581b6-cd03-4fb2-bb0a-9adebc49fa89 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8056e60ccbac44cdb0a2b9bd66e8e745', 'request_id': 'req-971581b6-cd03-4fb2-bb0a-9adebc49fa89'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:25:41.741417 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-971581b6-cd03-4fb2-bb0a-9adebc49fa89 admin admin] The token cache is being invalidate because project 8056e60ccbac44cdb0a2b9bd66e8e745 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80382) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 09:25:41.741708 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-971581b6-cd03-4fb2-bb0a-9adebc49fa89 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:25:41.742347 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-971581b6-cd03-4fb2-bb0a-9adebc49fa89 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:25:41.743937 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1368/2710] 10.222.0.26 () {58 vars in 1232 bytes} [Thu Jul 14 09:25:41 2022] DELETE /identity/v3/projects/8056e60ccbac44cdb0a2b9bd66e8e745 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:25:41.749131 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e947c5-e6d4-4503-900a-ad0f6ac378ba None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:41.749455 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e947c5-e6d4-4503-900a-ad0f6ac378ba None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:41.749723 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e947c5-e6d4-4503-900a-ad0f6ac378ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:41.819325 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-82e947c5-e6d4-4503-900a-ad0f6ac378ba None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:25:41.820685 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-82e947c5-e6d4-4503-900a-ad0f6ac378ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:25:41.881427 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1343/2711] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:25:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:25:41.890787 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b7c75a-ef2b-46e4-851e-bc2f3547e1c9 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:41.891008 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b7c75a-ef2b-46e4-851e-bc2f3547e1c9 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:41.891359 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b7c75a-ef2b-46e4-851e-bc2f3547e1c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:41.911731 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-b7b7c75a-ef2b-46e4-851e-bc2f3547e1c9 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:25:41.912944 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-b7b7c75a-ef2b-46e4-851e-bc2f3547e1c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:25:41.921503 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1369/2712] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:25:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 09:25:41.926304 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e947c5-e6d4-4503-900a-ad0f6ac378ba None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:25:41.938708 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e947c5-e6d4-4503-900a-ad0f6ac378ba None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:25:41.946444 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1008924b-f767-4ea0-aad1-6834d0709b4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:25:41.947474 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1008924b-f767-4ea0-aad1-6834d0709b4b admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:41.947757 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1008924b-f767-4ea0-aad1-6834d0709b4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:41.948021 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1008924b-f767-4ea0-aad1-6834d0709b4b admin admin] PATH_INFO: `/v3/domains` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:41.949076 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1008924b-f767-4ea0-aad1-6834d0709b4b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:25:41.949497 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1008924b-f767-4ea0-aad1-6834d0709b4b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:25:41.952102 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1008924b-f767-4ea0-aad1-6834d0709b4b admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:25:41.966906 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1344/2713] 10.222.0.26 () {58 vars in 1152 bytes} [Thu Jul 14 09:25:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:25:41.971638 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b7c75a-ef2b-46e4-851e-bc2f3547e1c9 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:25:41.981572 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b7c75a-ef2b-46e4-851e-bc2f3547e1c9 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:25:41.987036 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a487e63-404f-43c1-8d83-e41f1f8c96a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:25:41.987915 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a487e63-404f-43c1-8d83-e41f1f8c96a8 admin admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:41.988231 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a487e63-404f-43c1-8d83-e41f1f8c96a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:41.988520 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a487e63-404f-43c1-8d83-e41f1f8c96a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:41.989309 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a487e63-404f-43c1-8d83-e41f1f8c96a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:25:41.991368 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a487e63-404f-43c1-8d83-e41f1f8c96a8 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:25:42.002292 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:25:42.033073 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1370/2714] 10.222.0.26 () {60 vars in 1152 bytes} [Thu Jul 14 09:25:41 2022] POST /identity/v3/projects => generated 385 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 09:25:42.033954 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1008924b-f767-4ea0-aad1-6834d0709b4b admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:25:42.045806 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1008924b-f767-4ea0-aad1-6834d0709b4b admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:25:42.074811 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:25:42.110090 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e3bdb0-283e-48c9-80c5-4f5faa1f7781 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:25:42.110760 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e3bdb0-283e-48c9-80c5-4f5faa1f7781 admin admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:42.111043 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e3bdb0-283e-48c9-80c5-4f5faa1f7781 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:42.111352 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e3bdb0-283e-48c9-80c5-4f5faa1f7781 admin admin] PATH_INFO: `/v3/users` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:42.112061 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e3bdb0-283e-48c9-80c5-4f5faa1f7781 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:25:42.113983 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e3bdb0-283e-48c9-80c5-4f5faa1f7781 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:25:42.135165 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1345/2715] 10.222.0.26 () {60 vars in 1143 bytes} [Thu Jul 14 09:25:42 2022] POST /identity/v3/users => generated 293 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 09:25:42.139386 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a487e63-404f-43c1-8d83-e41f1f8c96a8 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:25:42.150697 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a487e63-404f-43c1-8d83-e41f1f8c96a8 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:25:42.155404 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cb25a4-c124-4955-9579-5b1985f97e8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:25:42.156083 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cb25a4-c124-4955-9579-5b1985f97e8b admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:42.156434 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cb25a4-c124-4955-9579-5b1985f97e8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:42.156695 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cb25a4-c124-4955-9579-5b1985f97e8b admin admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:42.157460 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cb25a4-c124-4955-9579-5b1985f97e8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:25:42.159173 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cb25a4-c124-4955-9579-5b1985f97e8b admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:25:42.174977 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1371/2716] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jul 14 09:25:42 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 09:25:42.176769 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e3bdb0-283e-48c9-80c5-4f5faa1f7781 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:25:42.189274 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e3bdb0-283e-48c9-80c5-4f5faa1f7781 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:25:42.197319 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec960ea-f1df-4200-8911-a243b7105b21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:25:42.198079 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec960ea-f1df-4200-8911-a243b7105b21 admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:42.198362 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec960ea-f1df-4200-8911-a243b7105b21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:42.198662 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec960ea-f1df-4200-8911-a243b7105b21 admin admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:42.199537 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec960ea-f1df-4200-8911-a243b7105b21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:25:42.201235 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec960ea-f1df-4200-8911-a243b7105b21 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:25:42.213023 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1346/2717] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jul 14 09:25:42 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 09:25:42.217816 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cb25a4-c124-4955-9579-5b1985f97e8b admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:25:42.232313 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cb25a4-c124-4955-9579-5b1985f97e8b admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:25:42.237457 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff2fff4-b67e-4a67-b762-b86781202032 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:25:42.238385 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff2fff4-b67e-4a67-b762-b86781202032 admin admin] REQUEST_METHOD: `PUT` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:42.238665 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff2fff4-b67e-4a67-b762-b86781202032 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:42.238945 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff2fff4-b67e-4a67-b762-b86781202032 admin admin] PATH_INFO: `/v3/projects/e7ff42bb028e4def83f7b70ba39076ae/users/d41abfe5e7ae4948adc1b8300daff8dc/roles/d1691322ce6a4edab7ce722edf0f2988` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:42.257752 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff2fff4-b67e-4a67-b762-b86781202032 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7ff42bb028e4def83f7b70ba39076ae, user_id=d41abfe5e7ae4948adc1b8300daff8dc, role_id=d1691322ce6a4edab7ce722edf0f2988)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:25:42.260421 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff2fff4-b67e-4a67-b762-b86781202032 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:25:42.277081 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1372/2718] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jul 14 09:25:42 2022] PUT /identity/v3/projects/e7ff42bb028e4def83f7b70ba39076ae/users/d41abfe5e7ae4948adc1b8300daff8dc/roles/d1691322ce6a4edab7ce722edf0f2988 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:25:42.288079 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcedc77-17e1-4ae5-9cd6-19937ba0a069 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:42.288560 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcedc77-17e1-4ae5-9cd6-19937ba0a069 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:42.288678 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcedc77-17e1-4ae5-9cd6-19937ba0a069 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:42.318324 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-dfcedc77-17e1-4ae5-9cd6-19937ba0a069 None None] MFA Rules not processed for user `d41abfe5e7ae4948adc1b8300daff8dc`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:25:42.319679 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-dfcedc77-17e1-4ae5-9cd6-19937ba0a069 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:25:42.398503 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1347/2719] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:25:42 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:25:42.403537 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff2fff4-b67e-4a67-b762-b86781202032 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:25:42.470947 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff2fff4-b67e-4a67-b762-b86781202032 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:25:42.476033 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54b175a-4e3c-4232-a6c8-aefe3a2f18ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:25:42.476716 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54b175a-4e3c-4232-a6c8-aefe3a2f18ea admin admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:42.477061 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54b175a-4e3c-4232-a6c8-aefe3a2f18ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:42.477255 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54b175a-4e3c-4232-a6c8-aefe3a2f18ea admin admin] PATH_INFO: `/v3/projects` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:42.477979 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54b175a-4e3c-4232-a6c8-aefe3a2f18ea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:25:42.479975 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54b175a-4e3c-4232-a6c8-aefe3a2f18ea admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:25:42.508125 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1373/2720] 10.222.0.26 () {60 vars in 1152 bytes} [Thu Jul 14 09:25:42 2022] POST /identity/v3/projects => generated 387 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 09:25:42.512240 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcedc77-17e1-4ae5-9cd6-19937ba0a069 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:25:42.524185 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcedc77-17e1-4ae5-9cd6-19937ba0a069 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:25:42.583674 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cf792b-f0b9-444a-88fb-5aafc2a63275 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:25:42.584354 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cf792b-f0b9-444a-88fb-5aafc2a63275 admin admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:42.584645 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cf792b-f0b9-444a-88fb-5aafc2a63275 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:42.584927 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cf792b-f0b9-444a-88fb-5aafc2a63275 admin admin] PATH_INFO: `/v3/users` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:42.585654 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cf792b-f0b9-444a-88fb-5aafc2a63275 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:25:42.587672 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cf792b-f0b9-444a-88fb-5aafc2a63275 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:25:42.615072 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1348/2721] 10.222.0.26 () {60 vars in 1143 bytes} [Thu Jul 14 09:25:42 2022] POST /identity/v3/users => generated 300 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 09:25:42.619494 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54b175a-4e3c-4232-a6c8-aefe3a2f18ea admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:25:42.630898 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54b175a-4e3c-4232-a6c8-aefe3a2f18ea admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:25:42.637947 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed4c536-63b8-45ab-a3ce-0636fa062566 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:25:42.638625 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed4c536-63b8-45ab-a3ce-0636fa062566 admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:42.638913 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed4c536-63b8-45ab-a3ce-0636fa062566 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:42.639216 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed4c536-63b8-45ab-a3ce-0636fa062566 admin admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:42.639957 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed4c536-63b8-45ab-a3ce-0636fa062566 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:25:42.641631 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed4c536-63b8-45ab-a3ce-0636fa062566 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:25:42.653746 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1374/2722] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jul 14 09:25:42 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 09:25:42.658176 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cf792b-f0b9-444a-88fb-5aafc2a63275 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:25:42.668744 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cf792b-f0b9-444a-88fb-5aafc2a63275 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:25:42.674670 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd3444d-f0de-46a6-8c32-664efa240378 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:25:42.675603 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd3444d-f0de-46a6-8c32-664efa240378 admin admin] REQUEST_METHOD: `PUT` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:42.675882 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd3444d-f0de-46a6-8c32-664efa240378 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:42.676161 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd3444d-f0de-46a6-8c32-664efa240378 admin admin] PATH_INFO: `/v3/projects/f9438f81e050419e98e73641562f2aa4/users/b26b5cc7fc61415abbde30a630adde7a/roles/7b549d58d1724395b343609d895c80d0` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:42.691899 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd3444d-f0de-46a6-8c32-664efa240378 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9438f81e050419e98e73641562f2aa4, user_id=b26b5cc7fc61415abbde30a630adde7a, role_id=7b549d58d1724395b343609d895c80d0)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:25:42.694555 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd3444d-f0de-46a6-8c32-664efa240378 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:25:42.709839 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1349/2723] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jul 14 09:25:42 2022] PUT /identity/v3/projects/f9438f81e050419e98e73641562f2aa4/users/b26b5cc7fc61415abbde30a630adde7a/roles/7b549d58d1724395b343609d895c80d0 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:25:42.713780 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed4c536-63b8-45ab-a3ce-0636fa062566 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:25:42.724364 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed4c536-63b8-45ab-a3ce-0636fa062566 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:25:42.783373 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561e35e5-01e4-4d63-948d-a7ccd4acac46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:25:42.784138 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561e35e5-01e4-4d63-948d-a7ccd4acac46 admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:42.784417 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561e35e5-01e4-4d63-948d-a7ccd4acac46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:42.784692 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561e35e5-01e4-4d63-948d-a7ccd4acac46 admin admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:42.785425 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561e35e5-01e4-4d63-948d-a7ccd4acac46 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:25:42.787056 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561e35e5-01e4-4d63-948d-a7ccd4acac46 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:25:42.797730 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1375/2724] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jul 14 09:25:42 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 09:25:42.802161 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd3444d-f0de-46a6-8c32-664efa240378 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:25:42.807843 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:25:42.813850 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd3444d-f0de-46a6-8c32-664efa240378 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:25:42.821273 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1469d108-a50f-4775-ad70-f0a07f65d8b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:25:42.822251 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1469d108-a50f-4775-ad70-f0a07f65d8b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:42.822527 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1469d108-a50f-4775-ad70-f0a07f65d8b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:42.822807 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1469d108-a50f-4775-ad70-f0a07f65d8b0 admin admin] PATH_INFO: `/v3/projects/f9438f81e050419e98e73641562f2aa4/users/b26b5cc7fc61415abbde30a630adde7a/roles/7b549d58d1724395b343609d895c80d0` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:42.824982 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1469d108-a50f-4775-ad70-f0a07f65d8b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9438f81e050419e98e73641562f2aa4, user_id=b26b5cc7fc61415abbde30a630adde7a, role_id=7b549d58d1724395b343609d895c80d0)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:25:42.830039 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1469d108-a50f-4775-ad70-f0a07f65d8b0 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:25:42.842949 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1350/2725] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jul 14 09:25:42 2022] PUT /identity/v3/projects/f9438f81e050419e98e73641562f2aa4/users/b26b5cc7fc61415abbde30a630adde7a/roles/7b549d58d1724395b343609d895c80d0 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:25:42.853894 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58830400-f61d-420c-bf37-9e827987c21d None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:42.854031 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58830400-f61d-420c-bf37-9e827987c21d None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:42.854301 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58830400-f61d-420c-bf37-9e827987c21d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:42.884786 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-58830400-f61d-420c-bf37-9e827987c21d None None] MFA Rules not processed for user `b26b5cc7fc61415abbde30a630adde7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:25:42.886152 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-58830400-f61d-420c-bf37-9e827987c21d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:25:42.900352 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:25:42.959906 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1376/2726] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:25:42 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:25:42.972696 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1469d108-a50f-4775-ad70-f0a07f65d8b0 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:25:42.974561 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-1469d108-a50f-4775-ad70-f0a07f65d8b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:25:43.055745 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1469d108-a50f-4775-ad70-f0a07f65d8b0 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:25:43.060165 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82ff5ea-cedc-4829-a43c-196d3f34b09b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e76f1952a5b146808844b9f2b7395035', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:25:43.060823 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82ff5ea-cedc-4829-a43c-196d3f34b09b service cinder] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:43.061225 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82ff5ea-cedc-4829-a43c-196d3f34b09b service cinder] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:43.061320 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82ff5ea-cedc-4829-a43c-196d3f34b09b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:43.071652 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82ff5ea-cedc-4829-a43c-196d3f34b09b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:25:43.073120 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82ff5ea-cedc-4829-a43c-196d3f34b09b service cinder] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:25:43.083238 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1351/2727] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:25:42 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:25:43.152129 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58830400-f61d-420c-bf37-9e827987c21d None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:25:43.225440 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58830400-f61d-420c-bf37-9e827987c21d None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:25:43.230720 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f9a648-0937-46d4-87c6-0741e9cdd261 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:25:43.231650 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f9a648-0937-46d4-87c6-0741e9cdd261 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:43.231928 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f9a648-0937-46d4-87c6-0741e9cdd261 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:43.232195 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f9a648-0937-46d4-87c6-0741e9cdd261 admin admin] PATH_INFO: `/v3/users/d41abfe5e7ae4948adc1b8300daff8dc` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:43.233406 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f9a648-0937-46d4-87c6-0741e9cdd261 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d41abfe5e7ae4948adc1b8300daff8dc)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:25:43.235457 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f9a648-0937-46d4-87c6-0741e9cdd261 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:25:43.286350 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-a3f9a648-0937-46d4-87c6-0741e9cdd261 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd41abfe5e7ae4948adc1b8300daff8dc'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:25:43.287000 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-a3f9a648-0937-46d4-87c6-0741e9cdd261 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd41abfe5e7ae4948adc1b8300daff8dc'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:25:43.299788 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-a3f9a648-0937-46d4-87c6-0741e9cdd261 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd41abfe5e7ae4948adc1b8300daff8dc'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:25:43.307794 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-a3f9a648-0937-46d4-87c6-0741e9cdd261 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd41abfe5e7ae4948adc1b8300daff8dc'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:25:43.308549 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-a3f9a648-0937-46d4-87c6-0741e9cdd261 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd41abfe5e7ae4948adc1b8300daff8dc'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:25:43.321895 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1377/2728] 10.222.0.26 () {58 vars in 1223 bytes} [Thu Jul 14 09:25:43 2022] DELETE /identity/v3/users/d41abfe5e7ae4948adc1b8300daff8dc => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:25:43.326420 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00f24b7-831a-472f-b261-561bc6b60f05 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:43.327330 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00f24b7-831a-472f-b261-561bc6b60f05 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:43.327613 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00f24b7-831a-472f-b261-561bc6b60f05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:43.346907 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-f00f24b7-831a-472f-b261-561bc6b60f05 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:25:43.348170 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-f00f24b7-831a-472f-b261-561bc6b60f05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:25:43.414820 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1352/2729] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:25:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 09:25:43.425298 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f9a648-0937-46d4-87c6-0741e9cdd261 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:25:43.426890 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-a3f9a648-0937-46d4-87c6-0741e9cdd261 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:25:43.464760 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:25:43.496372 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f9a648-0937-46d4-87c6-0741e9cdd261 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:25:43.500437 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16eace4d-a8ab-4f01-9a5c-9893497754eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '952911c492444ffea4b211a24b564117', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:25:43.501182 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16eace4d-a8ab-4f01-9a5c-9893497754eb service neutron] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:43.501551 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16eace4d-a8ab-4f01-9a5c-9893497754eb service neutron] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:43.501878 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16eace4d-a8ab-4f01-9a5c-9893497754eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:43.516013 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16eace4d-a8ab-4f01-9a5c-9893497754eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:25:43.519069 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16eace4d-a8ab-4f01-9a5c-9893497754eb service neutron] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:25:43.534593 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1378/2730] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:25:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:25:43.822701 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00f24b7-831a-472f-b261-561bc6b60f05 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:25:43.825104 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-f00f24b7-831a-472f-b261-561bc6b60f05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:25:43.838417 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00f24b7-831a-472f-b261-561bc6b60f05 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:25:43.843789 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b77a8a-5df5-4e31-b976-340c196345f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:25:43.844684 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b77a8a-5df5-4e31-b976-340c196345f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:43.844976 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b77a8a-5df5-4e31-b976-340c196345f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:43.845259 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b77a8a-5df5-4e31-b976-340c196345f8 admin admin] PATH_INFO: `/v3/projects/e7ff42bb028e4def83f7b70ba39076ae` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:43.846469 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b77a8a-5df5-4e31-b976-340c196345f8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e7ff42bb028e4def83f7b70ba39076ae)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:25:43.848418 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b77a8a-5df5-4e31-b976-340c196345f8 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:25:43.899045 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-d6b77a8a-5df5-4e31-b976-340c196345f8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e7ff42bb028e4def83f7b70ba39076ae', 'request_id': 'req-d6b77a8a-5df5-4e31-b976-340c196345f8'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:25:43.905673 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-d6b77a8a-5df5-4e31-b976-340c196345f8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e7ff42bb028e4def83f7b70ba39076ae', 'request_id': 'req-d6b77a8a-5df5-4e31-b976-340c196345f8'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:25:43.911416 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-d6b77a8a-5df5-4e31-b976-340c196345f8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e7ff42bb028e4def83f7b70ba39076ae', 'request_id': 'req-d6b77a8a-5df5-4e31-b976-340c196345f8'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:25:43.923987 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:25:43.928349 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-d6b77a8a-5df5-4e31-b976-340c196345f8 admin admin] The token cache is being invalidate because project e7ff42bb028e4def83f7b70ba39076ae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80381) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 09:25:43.928894 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-d6b77a8a-5df5-4e31-b976-340c196345f8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:25:43.929933 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-d6b77a8a-5df5-4e31-b976-340c196345f8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:25:43.931922 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1353/2731] 10.222.0.26 () {58 vars in 1232 bytes} [Thu Jul 14 09:25:43 2022] DELETE /identity/v3/projects/e7ff42bb028e4def83f7b70ba39076ae => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:25:43.935782 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16eace4d-a8ab-4f01-9a5c-9893497754eb service neutron] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:25:43.945966 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16eace4d-a8ab-4f01-9a5c-9893497754eb service neutron] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:25:44.006493 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ca5713-bbd9-437c-9092-7da6fcc99462 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:25:44.007571 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ca5713-bbd9-437c-9092-7da6fcc99462 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:44.007859 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ca5713-bbd9-437c-9092-7da6fcc99462 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:44.008132 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ca5713-bbd9-437c-9092-7da6fcc99462 admin admin] PATH_INFO: `/v3/users/b26b5cc7fc61415abbde30a630adde7a` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:44.009512 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ca5713-bbd9-437c-9092-7da6fcc99462 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b26b5cc7fc61415abbde30a630adde7a)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:25:44.011531 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ca5713-bbd9-437c-9092-7da6fcc99462 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:25:44.059489 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-b8ca5713-bbd9-437c-9092-7da6fcc99462 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b26b5cc7fc61415abbde30a630adde7a'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:25:44.060154 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-b8ca5713-bbd9-437c-9092-7da6fcc99462 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b26b5cc7fc61415abbde30a630adde7a'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:25:44.073168 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-b8ca5713-bbd9-437c-9092-7da6fcc99462 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b26b5cc7fc61415abbde30a630adde7a'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:25:44.082098 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-b8ca5713-bbd9-437c-9092-7da6fcc99462 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b26b5cc7fc61415abbde30a630adde7a'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:25:44.083355 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-b8ca5713-bbd9-437c-9092-7da6fcc99462 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b26b5cc7fc61415abbde30a630adde7a'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:25:44.099972 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1379/2732] 10.222.0.26 () {58 vars in 1223 bytes} [Thu Jul 14 09:25:43 2022] DELETE /identity/v3/users/b26b5cc7fc61415abbde30a630adde7a => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:25:44.375062 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b77a8a-5df5-4e31-b976-340c196345f8 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:25:44.376730 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-d6b77a8a-5df5-4e31-b976-340c196345f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:25:44.390181 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b77a8a-5df5-4e31-b976-340c196345f8 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:25:44.451336 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273f81fc-4bfe-42e2-a161-c043a0a2ec33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:25:44.452204 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273f81fc-4bfe-42e2-a161-c043a0a2ec33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:44.452488 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273f81fc-4bfe-42e2-a161-c043a0a2ec33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:44.452764 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273f81fc-4bfe-42e2-a161-c043a0a2ec33 admin admin] PATH_INFO: `/v3/projects/f9438f81e050419e98e73641562f2aa4` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:44.454139 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273f81fc-4bfe-42e2-a161-c043a0a2ec33 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f9438f81e050419e98e73641562f2aa4)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:25:44.456090 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273f81fc-4bfe-42e2-a161-c043a0a2ec33 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:25:44.512161 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-273f81fc-4bfe-42e2-a161-c043a0a2ec33 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f9438f81e050419e98e73641562f2aa4', 'request_id': 'req-273f81fc-4bfe-42e2-a161-c043a0a2ec33'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:25:44.519836 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-273f81fc-4bfe-42e2-a161-c043a0a2ec33 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f9438f81e050419e98e73641562f2aa4', 'request_id': 'req-273f81fc-4bfe-42e2-a161-c043a0a2ec33'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:25:44.525668 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-273f81fc-4bfe-42e2-a161-c043a0a2ec33 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f9438f81e050419e98e73641562f2aa4', 'request_id': 'req-273f81fc-4bfe-42e2-a161-c043a0a2ec33'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:25:44.544636 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-273f81fc-4bfe-42e2-a161-c043a0a2ec33 admin admin] The token cache is being invalidate because project f9438f81e050419e98e73641562f2aa4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80381) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 09:25:44.544924 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-273f81fc-4bfe-42e2-a161-c043a0a2ec33 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:25:44.545555 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-273f81fc-4bfe-42e2-a161-c043a0a2ec33 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:25:44.547177 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1354/2733] 10.222.0.26 () {58 vars in 1232 bytes} [Thu Jul 14 09:25:44 2022] DELETE /identity/v3/projects/f9438f81e050419e98e73641562f2aa4 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:25:44.552107 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f4685d-1510-4553-b531-11ce22542297 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:44.552393 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f4685d-1510-4553-b531-11ce22542297 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:44.552667 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f4685d-1510-4553-b531-11ce22542297 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:44.571633 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-66f4685d-1510-4553-b531-11ce22542297 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:25:44.572835 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-66f4685d-1510-4553-b531-11ce22542297 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:25:44.623014 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:25:44.638502 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1380/2734] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:25:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 09:25:44.646556 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adc322d-3565-43ab-97de-4420b0e08d18 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:44.646844 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adc322d-3565-43ab-97de-4420b0e08d18 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:44.647117 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adc322d-3565-43ab-97de-4420b0e08d18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:44.666717 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-0adc322d-3565-43ab-97de-4420b0e08d18 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:25:44.667943 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-0adc322d-3565-43ab-97de-4420b0e08d18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:25:44.677043 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1355/2735] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:25:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:25:44.681285 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f4685d-1510-4553-b531-11ce22542297 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:25:44.691241 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f4685d-1510-4553-b531-11ce22542297 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:25:44.696351 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afcfe10-846c-47c8-909c-945ee1fbedc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:25:44.697000 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afcfe10-846c-47c8-909c-945ee1fbedc8 admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:44.697287 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afcfe10-846c-47c8-909c-945ee1fbedc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:44.697561 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afcfe10-846c-47c8-909c-945ee1fbedc8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:44.698287 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afcfe10-846c-47c8-909c-945ee1fbedc8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:25:44.698590 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afcfe10-846c-47c8-909c-945ee1fbedc8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:25:44.700220 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afcfe10-846c-47c8-909c-945ee1fbedc8 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:25:44.712660 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1381/2736] 10.222.0.26 () {58 vars in 1152 bytes} [Thu Jul 14 09:25:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:25:44.716799 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adc322d-3565-43ab-97de-4420b0e08d18 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:25:44.726375 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adc322d-3565-43ab-97de-4420b0e08d18 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:25:44.731762 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca727b77-3624-4371-a822-2ddb5e6e124f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:25:44.732406 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca727b77-3624-4371-a822-2ddb5e6e124f admin admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:44.732691 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca727b77-3624-4371-a822-2ddb5e6e124f admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:44.734180 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca727b77-3624-4371-a822-2ddb5e6e124f admin admin] PATH_INFO: `/v3/projects` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:44.734180 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca727b77-3624-4371-a822-2ddb5e6e124f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:25:44.735628 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca727b77-3624-4371-a822-2ddb5e6e124f admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:25:44.762586 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1356/2737] 10.222.0.26 () {60 vars in 1152 bytes} [Thu Jul 14 09:25:44 2022] POST /identity/v3/projects => generated 379 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 09:25:44.766658 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afcfe10-846c-47c8-909c-945ee1fbedc8 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:25:44.776960 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afcfe10-846c-47c8-909c-945ee1fbedc8 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:25:44.834639 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775b9312-1ee6-45fe-b755-bccd29e50e43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:25:44.835360 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775b9312-1ee6-45fe-b755-bccd29e50e43 admin admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:44.835645 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775b9312-1ee6-45fe-b755-bccd29e50e43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:44.835920 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775b9312-1ee6-45fe-b755-bccd29e50e43 admin admin] PATH_INFO: `/v3/users` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:44.836610 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775b9312-1ee6-45fe-b755-bccd29e50e43 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:25:44.838614 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775b9312-1ee6-45fe-b755-bccd29e50e43 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:25:44.862175 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1382/2738] 10.222.0.26 () {60 vars in 1143 bytes} [Thu Jul 14 09:25:44 2022] POST /identity/v3/users => generated 290 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 09:25:44.866484 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca727b77-3624-4371-a822-2ddb5e6e124f admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:25:44.877173 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca727b77-3624-4371-a822-2ddb5e6e124f admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:25:44.882338 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4060b8-a985-4bcf-8915-6cae15b7034c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:25:44.883021 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4060b8-a985-4bcf-8915-6cae15b7034c admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:44.883348 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4060b8-a985-4bcf-8915-6cae15b7034c admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:44.883627 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4060b8-a985-4bcf-8915-6cae15b7034c admin admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:44.884362 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4060b8-a985-4bcf-8915-6cae15b7034c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:25:44.885972 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4060b8-a985-4bcf-8915-6cae15b7034c admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:25:44.897264 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1357/2739] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jul 14 09:25:44 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 09:25:44.901426 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775b9312-1ee6-45fe-b755-bccd29e50e43 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:25:44.912266 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775b9312-1ee6-45fe-b755-bccd29e50e43 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:25:44.917430 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0e368a-4427-406e-b787-e42bd6a3f9af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:25:44.918134 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0e368a-4427-406e-b787-e42bd6a3f9af admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:44.918417 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0e368a-4427-406e-b787-e42bd6a3f9af admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:44.918683 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0e368a-4427-406e-b787-e42bd6a3f9af admin admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:44.919511 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0e368a-4427-406e-b787-e42bd6a3f9af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:25:44.921147 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0e368a-4427-406e-b787-e42bd6a3f9af admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:25:44.931595 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1383/2740] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jul 14 09:25:44 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 09:25:44.935905 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4060b8-a985-4bcf-8915-6cae15b7034c admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:25:44.946894 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4060b8-a985-4bcf-8915-6cae15b7034c admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:25:44.952494 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0fac0e-87ef-4fbc-b44b-c33ecfa2c120 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:25:44.953595 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0fac0e-87ef-4fbc-b44b-c33ecfa2c120 admin admin] REQUEST_METHOD: `PUT` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:44.953878 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0fac0e-87ef-4fbc-b44b-c33ecfa2c120 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:44.954149 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0fac0e-87ef-4fbc-b44b-c33ecfa2c120 admin admin] PATH_INFO: `/v3/projects/86581a69b6e94e328ca6cf3d710b7685/users/7caa0a76a4ef46ffa392f13dc5de9b82/roles/d1691322ce6a4edab7ce722edf0f2988` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:44.970260 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0fac0e-87ef-4fbc-b44b-c33ecfa2c120 admin admin] RBAC: Authorizing `identity:create_grant(project_id=86581a69b6e94e328ca6cf3d710b7685, user_id=7caa0a76a4ef46ffa392f13dc5de9b82, role_id=d1691322ce6a4edab7ce722edf0f2988)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:25:44.972952 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0fac0e-87ef-4fbc-b44b-c33ecfa2c120 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:25:44.988328 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1358/2741] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jul 14 09:25:44 2022] PUT /identity/v3/projects/86581a69b6e94e328ca6cf3d710b7685/users/7caa0a76a4ef46ffa392f13dc5de9b82/roles/d1691322ce6a4edab7ce722edf0f2988 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 09:25:44.999236 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb0cb05-d256-4597-bbfa-713bed06f95c None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:44.999766 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb0cb05-d256-4597-bbfa-713bed06f95c None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:44.999766 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb0cb05-d256-4597-bbfa-713bed06f95c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:45.037695 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-8eb0cb05-d256-4597-bbfa-713bed06f95c None None] MFA Rules not processed for user `7caa0a76a4ef46ffa392f13dc5de9b82`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:25:45.038878 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-8eb0cb05-d256-4597-bbfa-713bed06f95c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:25:45.113459 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1384/2742] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:25:44 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:25:45.118313 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0fac0e-87ef-4fbc-b44b-c33ecfa2c120 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:25:45.149918 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:25:45.185707 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0fac0e-87ef-4fbc-b44b-c33ecfa2c120 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:25:45.190539 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16be0e21-afac-433c-ac3f-0833710199db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:25:45.191248 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16be0e21-afac-433c-ac3f-0833710199db admin admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:45.191566 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16be0e21-afac-433c-ac3f-0833710199db admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:45.191847 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16be0e21-afac-433c-ac3f-0833710199db admin admin] PATH_INFO: `/v3/projects` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:45.192552 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16be0e21-afac-433c-ac3f-0833710199db admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:25:45.194472 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16be0e21-afac-433c-ac3f-0833710199db admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:25:45.222247 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1359/2743] 10.222.0.26 () {60 vars in 1152 bytes} [Thu Jul 14 09:25:45 2022] POST /identity/v3/projects => generated 381 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 09:25:45.226404 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb0cb05-d256-4597-bbfa-713bed06f95c None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:25:45.237903 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb0cb05-d256-4597-bbfa-713bed06f95c None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:25:45.285814 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:25:45.299732 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9189940-e7fe-4dec-85b8-0c4ddea4bf9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:25:45.300340 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9189940-e7fe-4dec-85b8-0c4ddea4bf9a admin admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:45.300617 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9189940-e7fe-4dec-85b8-0c4ddea4bf9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:45.300885 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9189940-e7fe-4dec-85b8-0c4ddea4bf9a admin admin] PATH_INFO: `/v3/users` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:45.301606 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9189940-e7fe-4dec-85b8-0c4ddea4bf9a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:25:45.303608 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9189940-e7fe-4dec-85b8-0c4ddea4bf9a admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:25:45.323530 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1385/2744] 10.222.0.26 () {60 vars in 1143 bytes} [Thu Jul 14 09:25:45 2022] POST /identity/v3/users => generated 297 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 09:25:45.327540 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16be0e21-afac-433c-ac3f-0833710199db admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:25:45.337967 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16be0e21-afac-433c-ac3f-0833710199db admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:25:45.342981 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075805a6-9204-4492-b346-c941a24067f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:25:45.343721 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075805a6-9204-4492-b346-c941a24067f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:45.344005 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075805a6-9204-4492-b346-c941a24067f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:45.344259 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075805a6-9204-4492-b346-c941a24067f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:45.345004 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075805a6-9204-4492-b346-c941a24067f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:25:45.346596 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075805a6-9204-4492-b346-c941a24067f2 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:25:45.357363 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1360/2745] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jul 14 09:25:45 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 09:25:45.361429 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9189940-e7fe-4dec-85b8-0c4ddea4bf9a admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:25:45.371614 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9189940-e7fe-4dec-85b8-0c4ddea4bf9a admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:25:45.376411 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad92392e-c1eb-4133-a05a-f023dd959a18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:25:45.377286 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad92392e-c1eb-4133-a05a-f023dd959a18 admin admin] REQUEST_METHOD: `PUT` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:45.377567 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad92392e-c1eb-4133-a05a-f023dd959a18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:45.377840 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad92392e-c1eb-4133-a05a-f023dd959a18 admin admin] PATH_INFO: `/v3/projects/7a6ac44258284eedb5f606f2965cbb2d/users/8dd62323627f4134aec045f329f997cd/roles/7b549d58d1724395b343609d895c80d0` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:45.393348 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad92392e-c1eb-4133-a05a-f023dd959a18 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a6ac44258284eedb5f606f2965cbb2d, user_id=8dd62323627f4134aec045f329f997cd, role_id=7b549d58d1724395b343609d895c80d0)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:25:45.396337 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad92392e-c1eb-4133-a05a-f023dd959a18 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:25:45.411797 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1386/2746] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jul 14 09:25:45 2022] PUT /identity/v3/projects/7a6ac44258284eedb5f606f2965cbb2d/users/8dd62323627f4134aec045f329f997cd/roles/7b549d58d1724395b343609d895c80d0 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:25:45.416294 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075805a6-9204-4492-b346-c941a24067f2 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:25:45.426700 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075805a6-9204-4492-b346-c941a24067f2 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:25:45.497476 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecc2d94-1f67-48a0-83aa-55402882a5b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:25:45.498140 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecc2d94-1f67-48a0-83aa-55402882a5b2 admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:45.498416 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecc2d94-1f67-48a0-83aa-55402882a5b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:45.498670 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecc2d94-1f67-48a0-83aa-55402882a5b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:45.499436 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecc2d94-1f67-48a0-83aa-55402882a5b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:25:45.501028 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecc2d94-1f67-48a0-83aa-55402882a5b2 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:25:45.511265 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1361/2747] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jul 14 09:25:45 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 09:25:45.515559 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad92392e-c1eb-4133-a05a-f023dd959a18 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:25:45.527352 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad92392e-c1eb-4133-a05a-f023dd959a18 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:25:45.533828 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557b64ee-450e-4a60-8523-8b3e57edd8be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:25:45.535385 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557b64ee-450e-4a60-8523-8b3e57edd8be admin admin] REQUEST_METHOD: `PUT` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:45.535805 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557b64ee-450e-4a60-8523-8b3e57edd8be admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:45.536250 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557b64ee-450e-4a60-8523-8b3e57edd8be admin admin] PATH_INFO: `/v3/projects/7a6ac44258284eedb5f606f2965cbb2d/users/8dd62323627f4134aec045f329f997cd/roles/7b549d58d1724395b343609d895c80d0` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:45.539584 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557b64ee-450e-4a60-8523-8b3e57edd8be admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a6ac44258284eedb5f606f2965cbb2d, user_id=8dd62323627f4134aec045f329f997cd, role_id=7b549d58d1724395b343609d895c80d0)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:25:45.544433 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557b64ee-450e-4a60-8523-8b3e57edd8be admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:25:45.562986 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1387/2748] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jul 14 09:25:45 2022] PUT /identity/v3/projects/7a6ac44258284eedb5f606f2965cbb2d/users/8dd62323627f4134aec045f329f997cd/roles/7b549d58d1724395b343609d895c80d0 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:25:45.579722 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab223bc8-23dc-4b4f-b860-2f625e915599 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:45.580011 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab223bc8-23dc-4b4f-b860-2f625e915599 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:45.580294 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab223bc8-23dc-4b4f-b860-2f625e915599 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:45.608540 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-ab223bc8-23dc-4b4f-b860-2f625e915599 None None] MFA Rules not processed for user `8dd62323627f4134aec045f329f997cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:25:45.609756 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-ab223bc8-23dc-4b4f-b860-2f625e915599 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:25:45.681976 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1362/2749] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:25:45 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:25:45.751423 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557b64ee-450e-4a60-8523-8b3e57edd8be admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:25:45.753590 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-557b64ee-450e-4a60-8523-8b3e57edd8be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:25:45.814773 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557b64ee-450e-4a60-8523-8b3e57edd8be admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:25:45.821009 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a56bee-0c6b-443a-90b5-4f03e0671a13 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e76f1952a5b146808844b9f2b7395035', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:25:45.821974 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a56bee-0c6b-443a-90b5-4f03e0671a13 service cinder] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:45.822265 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a56bee-0c6b-443a-90b5-4f03e0671a13 service cinder] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:45.822584 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a56bee-0c6b-443a-90b5-4f03e0671a13 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:45.836782 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a56bee-0c6b-443a-90b5-4f03e0671a13 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:25:45.839054 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a56bee-0c6b-443a-90b5-4f03e0671a13 service cinder] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:25:45.851938 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1388/2750] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:25:45 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:25:46.269405 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab223bc8-23dc-4b4f-b860-2f625e915599 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:25:46.299936 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:25:46.344674 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab223bc8-23dc-4b4f-b860-2f625e915599 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:25:46.349219 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fcc610-1136-4964-8537-12af8dd91ddb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:25:46.350452 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fcc610-1136-4964-8537-12af8dd91ddb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:46.350452 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fcc610-1136-4964-8537-12af8dd91ddb admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:46.350695 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fcc610-1136-4964-8537-12af8dd91ddb admin admin] PATH_INFO: `/v3/users/7caa0a76a4ef46ffa392f13dc5de9b82` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:46.351884 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fcc610-1136-4964-8537-12af8dd91ddb admin admin] RBAC: Authorizing `identity:delete_user(user_id=7caa0a76a4ef46ffa392f13dc5de9b82)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:25:46.353831 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fcc610-1136-4964-8537-12af8dd91ddb admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:25:46.408659 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-43fcc610-1136-4964-8537-12af8dd91ddb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7caa0a76a4ef46ffa392f13dc5de9b82'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:25:46.418721 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-43fcc610-1136-4964-8537-12af8dd91ddb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7caa0a76a4ef46ffa392f13dc5de9b82'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:25:46.419405 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-43fcc610-1136-4964-8537-12af8dd91ddb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7caa0a76a4ef46ffa392f13dc5de9b82'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:25:46.420067 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-43fcc610-1136-4964-8537-12af8dd91ddb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7caa0a76a4ef46ffa392f13dc5de9b82'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:25:46.428138 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-43fcc610-1136-4964-8537-12af8dd91ddb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7caa0a76a4ef46ffa392f13dc5de9b82'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:25:46.448587 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1363/2751] 10.222.0.26 () {58 vars in 1223 bytes} [Thu Jul 14 09:25:46 2022] DELETE /identity/v3/users/7caa0a76a4ef46ffa392f13dc5de9b82 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:25:46.453737 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b301b984-0ab4-4345-8d93-6cc1d9bd0621 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:46.454258 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b301b984-0ab4-4345-8d93-6cc1d9bd0621 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:46.454449 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b301b984-0ab4-4345-8d93-6cc1d9bd0621 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:46.475827 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-b301b984-0ab4-4345-8d93-6cc1d9bd0621 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:25:46.477102 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-b301b984-0ab4-4345-8d93-6cc1d9bd0621 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:25:46.549550 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1389/2752] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:25:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:25:46.559435 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fcc610-1136-4964-8537-12af8dd91ddb admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:25:46.561506 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-43fcc610-1136-4964-8537-12af8dd91ddb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:25:46.627975 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fcc610-1136-4964-8537-12af8dd91ddb admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:25:46.631994 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74af03d1-8220-450c-adb5-1975833a0b88 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '952911c492444ffea4b211a24b564117', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:25:46.632652 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74af03d1-8220-450c-adb5-1975833a0b88 service neutron] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:46.632940 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74af03d1-8220-450c-adb5-1975833a0b88 service neutron] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:46.633215 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74af03d1-8220-450c-adb5-1975833a0b88 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:46.642939 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74af03d1-8220-450c-adb5-1975833a0b88 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:25:46.644419 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74af03d1-8220-450c-adb5-1975833a0b88 service neutron] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:25:46.655234 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1364/2753] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:25:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:25:46.996514 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b301b984-0ab4-4345-8d93-6cc1d9bd0621 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:25:46.998169 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-b301b984-0ab4-4345-8d93-6cc1d9bd0621 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:25:47.015627 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b301b984-0ab4-4345-8d93-6cc1d9bd0621 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:25:47.021773 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef354dd-0e02-4ff0-be72-44e4e9c0974e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:25:47.022668 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef354dd-0e02-4ff0-be72-44e4e9c0974e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:47.022983 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef354dd-0e02-4ff0-be72-44e4e9c0974e admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:47.023402 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef354dd-0e02-4ff0-be72-44e4e9c0974e admin admin] PATH_INFO: `/v3/projects/86581a69b6e94e328ca6cf3d710b7685` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:47.024700 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef354dd-0e02-4ff0-be72-44e4e9c0974e admin admin] RBAC: Authorizing `identity:delete_project(project_id=86581a69b6e94e328ca6cf3d710b7685)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:25:47.026699 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef354dd-0e02-4ff0-be72-44e4e9c0974e admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:25:47.074253 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:25:47.083332 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-1ef354dd-0e02-4ff0-be72-44e4e9c0974e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '86581a69b6e94e328ca6cf3d710b7685', 'request_id': 'req-1ef354dd-0e02-4ff0-be72-44e4e9c0974e'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:25:47.093344 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-1ef354dd-0e02-4ff0-be72-44e4e9c0974e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '86581a69b6e94e328ca6cf3d710b7685', 'request_id': 'req-1ef354dd-0e02-4ff0-be72-44e4e9c0974e'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:25:47.100339 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-1ef354dd-0e02-4ff0-be72-44e4e9c0974e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '86581a69b6e94e328ca6cf3d710b7685', 'request_id': 'req-1ef354dd-0e02-4ff0-be72-44e4e9c0974e'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:25:47.121988 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-1ef354dd-0e02-4ff0-be72-44e4e9c0974e admin admin] The token cache is being invalidate because project 86581a69b6e94e328ca6cf3d710b7685 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80382) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 09:25:47.122922 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-1ef354dd-0e02-4ff0-be72-44e4e9c0974e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:25:47.123902 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-1ef354dd-0e02-4ff0-be72-44e4e9c0974e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:25:47.125636 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1390/2754] 10.222.0.26 () {58 vars in 1232 bytes} [Thu Jul 14 09:25:46 2022] DELETE /identity/v3/projects/86581a69b6e94e328ca6cf3d710b7685 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:25:47.129772 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74af03d1-8220-450c-adb5-1975833a0b88 service neutron] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:25:47.140250 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74af03d1-8220-450c-adb5-1975833a0b88 service neutron] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:25:47.186870 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:25:47.206067 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52452a0a-422c-4500-afab-b9e2853d172b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:25:47.207690 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52452a0a-422c-4500-afab-b9e2853d172b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:47.208056 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52452a0a-422c-4500-afab-b9e2853d172b admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:47.208477 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52452a0a-422c-4500-afab-b9e2853d172b admin admin] PATH_INFO: `/v3/users/8dd62323627f4134aec045f329f997cd` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:47.209764 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52452a0a-422c-4500-afab-b9e2853d172b admin admin] RBAC: Authorizing `identity:delete_user(user_id=8dd62323627f4134aec045f329f997cd)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:25:47.211723 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52452a0a-422c-4500-afab-b9e2853d172b admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:25:47.259098 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-52452a0a-422c-4500-afab-b9e2853d172b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8dd62323627f4134aec045f329f997cd'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:25:47.267765 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-52452a0a-422c-4500-afab-b9e2853d172b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8dd62323627f4134aec045f329f997cd'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:25:47.268363 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-52452a0a-422c-4500-afab-b9e2853d172b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8dd62323627f4134aec045f329f997cd'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:25:47.269024 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-52452a0a-422c-4500-afab-b9e2853d172b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8dd62323627f4134aec045f329f997cd'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:25:47.276300 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-52452a0a-422c-4500-afab-b9e2853d172b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8dd62323627f4134aec045f329f997cd'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:25:47.294014 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1365/2755] 10.222.0.26 () {58 vars in 1223 bytes} [Thu Jul 14 09:25:47 2022] DELETE /identity/v3/users/8dd62323627f4134aec045f329f997cd => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:25:47.587289 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef354dd-0e02-4ff0-be72-44e4e9c0974e admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:25:47.589915 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-1ef354dd-0e02-4ff0-be72-44e4e9c0974e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:25:47.606580 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef354dd-0e02-4ff0-be72-44e4e9c0974e admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:25:47.694057 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed69ee85-e8b9-4c52-909b-b720f76ad9c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:25:47.695583 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed69ee85-e8b9-4c52-909b-b720f76ad9c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:47.696208 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed69ee85-e8b9-4c52-909b-b720f76ad9c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:47.696512 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed69ee85-e8b9-4c52-909b-b720f76ad9c0 admin admin] PATH_INFO: `/v3/projects/7a6ac44258284eedb5f606f2965cbb2d` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:47.697831 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed69ee85-e8b9-4c52-909b-b720f76ad9c0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7a6ac44258284eedb5f606f2965cbb2d)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:25:47.699879 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed69ee85-e8b9-4c52-909b-b720f76ad9c0 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:25:47.762620 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-ed69ee85-e8b9-4c52-909b-b720f76ad9c0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a6ac44258284eedb5f606f2965cbb2d', 'request_id': 'req-ed69ee85-e8b9-4c52-909b-b720f76ad9c0'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:25:47.769262 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-ed69ee85-e8b9-4c52-909b-b720f76ad9c0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a6ac44258284eedb5f606f2965cbb2d', 'request_id': 'req-ed69ee85-e8b9-4c52-909b-b720f76ad9c0'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:25:47.775286 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-ed69ee85-e8b9-4c52-909b-b720f76ad9c0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a6ac44258284eedb5f606f2965cbb2d', 'request_id': 'req-ed69ee85-e8b9-4c52-909b-b720f76ad9c0'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:25:47.787994 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-ed69ee85-e8b9-4c52-909b-b720f76ad9c0 admin admin] The token cache is being invalidate because project 7a6ac44258284eedb5f606f2965cbb2d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80382) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 09:25:47.788287 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-ed69ee85-e8b9-4c52-909b-b720f76ad9c0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:25:47.788885 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-ed69ee85-e8b9-4c52-909b-b720f76ad9c0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:25:47.790435 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1391/2756] 10.222.0.26 () {58 vars in 1232 bytes} [Thu Jul 14 09:25:47 2022] DELETE /identity/v3/projects/7a6ac44258284eedb5f606f2965cbb2d => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:25:47.795575 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e42b263-16be-4f3f-9807-73f5d693f962 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:47.796054 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e42b263-16be-4f3f-9807-73f5d693f962 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:47.796171 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e42b263-16be-4f3f-9807-73f5d693f962 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:47.815833 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-6e42b263-16be-4f3f-9807-73f5d693f962 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:25:47.817062 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-6e42b263-16be-4f3f-9807-73f5d693f962 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:25:47.882753 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1366/2757] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:25:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 09:25:47.890926 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17b91b5-fa55-4d21-946e-615fe951b1d3 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:47.891223 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17b91b5-fa55-4d21-946e-615fe951b1d3 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:47.891515 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17b91b5-fa55-4d21-946e-615fe951b1d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:47.912575 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-b17b91b5-fa55-4d21-946e-615fe951b1d3 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:25:47.913704 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-b17b91b5-fa55-4d21-946e-615fe951b1d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:25:47.923134 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1392/2758] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:25:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:25:47.927587 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e42b263-16be-4f3f-9807-73f5d693f962 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:25:47.938238 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e42b263-16be-4f3f-9807-73f5d693f962 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:25:47.944189 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06d4641-62a3-47a4-962a-38fb40cbff56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:25:47.944840 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06d4641-62a3-47a4-962a-38fb40cbff56 admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:47.945123 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06d4641-62a3-47a4-962a-38fb40cbff56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:47.945392 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06d4641-62a3-47a4-962a-38fb40cbff56 admin admin] PATH_INFO: `/v3/domains` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:47.946116 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06d4641-62a3-47a4-962a-38fb40cbff56 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:25:47.946419 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06d4641-62a3-47a4-962a-38fb40cbff56 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:25:47.948055 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06d4641-62a3-47a4-962a-38fb40cbff56 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:25:47.960516 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1367/2759] 10.222.0.26 () {58 vars in 1152 bytes} [Thu Jul 14 09:25:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:25:47.964810 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17b91b5-fa55-4d21-946e-615fe951b1d3 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:25:47.974278 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17b91b5-fa55-4d21-946e-615fe951b1d3 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:25:47.980231 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d380929-4c2a-49eb-b5f4-10c2ecf5ff89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:25:47.980877 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d380929-4c2a-49eb-b5f4-10c2ecf5ff89 admin admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:47.981151 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d380929-4c2a-49eb-b5f4-10c2ecf5ff89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:47.981420 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d380929-4c2a-49eb-b5f4-10c2ecf5ff89 admin admin] PATH_INFO: `/v3/projects` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:47.982097 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d380929-4c2a-49eb-b5f4-10c2ecf5ff89 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:25:47.984018 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d380929-4c2a-49eb-b5f4-10c2ecf5ff89 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:25:48.012372 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1393/2760] 10.222.0.26 () {60 vars in 1152 bytes} [Thu Jul 14 09:25:47 2022] POST /identity/v3/projects => generated 373 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 09:25:48.016738 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06d4641-62a3-47a4-962a-38fb40cbff56 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:25:48.027558 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06d4641-62a3-47a4-962a-38fb40cbff56 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:25:48.079159 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:25:48.092547 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0590d7a2-d75a-4314-b69d-226fe95f49b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:25:48.093237 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0590d7a2-d75a-4314-b69d-226fe95f49b4 admin admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:48.093529 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0590d7a2-d75a-4314-b69d-226fe95f49b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:48.093857 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0590d7a2-d75a-4314-b69d-226fe95f49b4 admin admin] PATH_INFO: `/v3/users` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:48.094657 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0590d7a2-d75a-4314-b69d-226fe95f49b4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:25:48.097068 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0590d7a2-d75a-4314-b69d-226fe95f49b4 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:25:48.120480 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1368/2761] 10.222.0.26 () {60 vars in 1143 bytes} [Thu Jul 14 09:25:48 2022] POST /identity/v3/users => generated 287 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 09:25:48.124898 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d380929-4c2a-49eb-b5f4-10c2ecf5ff89 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:25:48.136506 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d380929-4c2a-49eb-b5f4-10c2ecf5ff89 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:25:48.141435 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f024630e-6cf1-4af3-9f59-b3c80f78834f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:25:48.142106 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f024630e-6cf1-4af3-9f59-b3c80f78834f admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:48.142385 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f024630e-6cf1-4af3-9f59-b3c80f78834f admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:48.142693 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f024630e-6cf1-4af3-9f59-b3c80f78834f admin admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:48.143460 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f024630e-6cf1-4af3-9f59-b3c80f78834f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:25:48.145390 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f024630e-6cf1-4af3-9f59-b3c80f78834f admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:25:48.156872 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1394/2762] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jul 14 09:25:48 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 09:25:48.161162 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0590d7a2-d75a-4314-b69d-226fe95f49b4 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:25:48.173321 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0590d7a2-d75a-4314-b69d-226fe95f49b4 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:25:48.179447 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294ffbbf-68bb-4adb-a440-02eef84ad8f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:25:48.180120 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294ffbbf-68bb-4adb-a440-02eef84ad8f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:48.180401 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294ffbbf-68bb-4adb-a440-02eef84ad8f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:48.180717 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294ffbbf-68bb-4adb-a440-02eef84ad8f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:48.181621 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294ffbbf-68bb-4adb-a440-02eef84ad8f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:25:48.183243 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294ffbbf-68bb-4adb-a440-02eef84ad8f0 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:25:48.195729 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1369/2763] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jul 14 09:25:48 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 09:25:48.200015 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f024630e-6cf1-4af3-9f59-b3c80f78834f admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:25:48.210631 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f024630e-6cf1-4af3-9f59-b3c80f78834f admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:25:48.217366 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3840e6-d9ba-4954-b2bd-2d0ca2c88b19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:25:48.218257 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3840e6-d9ba-4954-b2bd-2d0ca2c88b19 admin admin] REQUEST_METHOD: `PUT` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:48.218539 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3840e6-d9ba-4954-b2bd-2d0ca2c88b19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:48.218877 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3840e6-d9ba-4954-b2bd-2d0ca2c88b19 admin admin] PATH_INFO: `/v3/projects/e633ca032ee7404699455c896ad8a31b/users/ce70deb538b046d9b45cd804d13e373d/roles/d1691322ce6a4edab7ce722edf0f2988` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:48.236885 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3840e6-d9ba-4954-b2bd-2d0ca2c88b19 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e633ca032ee7404699455c896ad8a31b, user_id=ce70deb538b046d9b45cd804d13e373d, role_id=d1691322ce6a4edab7ce722edf0f2988)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:25:48.238962 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3840e6-d9ba-4954-b2bd-2d0ca2c88b19 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:25:48.241373 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:25:48.256914 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1395/2764] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jul 14 09:25:48 2022] PUT /identity/v3/projects/e633ca032ee7404699455c896ad8a31b/users/ce70deb538b046d9b45cd804d13e373d/roles/d1691322ce6a4edab7ce722edf0f2988 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:25:48.268730 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c789b4-b669-4851-8522-526d092a3cba None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:48.269085 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c789b4-b669-4851-8522-526d092a3cba None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:48.269447 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c789b4-b669-4851-8522-526d092a3cba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:48.303517 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-07c789b4-b669-4851-8522-526d092a3cba None None] MFA Rules not processed for user `ce70deb538b046d9b45cd804d13e373d`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:25:48.304712 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-07c789b4-b669-4851-8522-526d092a3cba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:25:48.368810 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1370/2765] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:25:48 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:25:48.380495 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3840e6-d9ba-4954-b2bd-2d0ca2c88b19 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:25:48.382695 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-fb3840e6-d9ba-4954-b2bd-2d0ca2c88b19 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:25:48.445156 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3840e6-d9ba-4954-b2bd-2d0ca2c88b19 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:25:48.455220 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2905e92-037d-4527-a71e-0a340395bd9b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e76f1952a5b146808844b9f2b7395035', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:25:48.455220 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2905e92-037d-4527-a71e-0a340395bd9b service cinder] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:48.455220 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2905e92-037d-4527-a71e-0a340395bd9b service cinder] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:48.455220 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2905e92-037d-4527-a71e-0a340395bd9b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:48.461970 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2905e92-037d-4527-a71e-0a340395bd9b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:25:48.463481 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2905e92-037d-4527-a71e-0a340395bd9b service cinder] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:25:48.473146 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1396/2766] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:25:48 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:25:48.976266 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c789b4-b669-4851-8522-526d092a3cba None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:25:48.977834 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-07c789b4-b669-4851-8522-526d092a3cba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:25:49.065204 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c789b4-b669-4851-8522-526d092a3cba None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:25:49.070853 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c463057e-c166-4b13-b345-dcd067fdb4af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:25:49.071772 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c463057e-c166-4b13-b345-dcd067fdb4af admin admin] REQUEST_METHOD: `DELETE` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:49.072060 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c463057e-c166-4b13-b345-dcd067fdb4af admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:49.072335 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c463057e-c166-4b13-b345-dcd067fdb4af admin admin] PATH_INFO: `/v3/users/8c43c7d53825406188e26ecb1e5dc15a` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:49.073645 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c463057e-c166-4b13-b345-dcd067fdb4af admin admin] RBAC: Authorizing `identity:delete_user(user_id=8c43c7d53825406188e26ecb1e5dc15a)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:25:49.075626 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c463057e-c166-4b13-b345-dcd067fdb4af admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:25:49.126284 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-c463057e-c166-4b13-b345-dcd067fdb4af admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8c43c7d53825406188e26ecb1e5dc15a'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:25:49.135093 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-c463057e-c166-4b13-b345-dcd067fdb4af admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8c43c7d53825406188e26ecb1e5dc15a'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:25:49.135859 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-c463057e-c166-4b13-b345-dcd067fdb4af admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8c43c7d53825406188e26ecb1e5dc15a'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:25:49.136758 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-c463057e-c166-4b13-b345-dcd067fdb4af admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8c43c7d53825406188e26ecb1e5dc15a'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:25:49.142525 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:25:49.145555 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-c463057e-c166-4b13-b345-dcd067fdb4af admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8c43c7d53825406188e26ecb1e5dc15a'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:25:49.166186 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1371/2767] 10.222.0.26 () {58 vars in 1223 bytes} [Thu Jul 14 09:25:48 2022] DELETE /identity/v3/users/8c43c7d53825406188e26ecb1e5dc15a => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:25:49.172940 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0458563d-d5b7-4b87-acda-580c6f708143 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:49.173462 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0458563d-d5b7-4b87-acda-580c6f708143 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:49.173573 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0458563d-d5b7-4b87-acda-580c6f708143 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:49.192399 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-0458563d-d5b7-4b87-acda-580c6f708143 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:25:49.193626 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-0458563d-d5b7-4b87-acda-580c6f708143 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:25:49.269604 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1397/2768] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:25:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:25:49.279940 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c463057e-c166-4b13-b345-dcd067fdb4af admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:25:49.281728 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-c463057e-c166-4b13-b345-dcd067fdb4af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:25:49.365894 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c463057e-c166-4b13-b345-dcd067fdb4af admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:25:49.370566 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063b191c-30b6-40d8-847f-c67913d6c579 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '952911c492444ffea4b211a24b564117', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:25:49.371666 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063b191c-30b6-40d8-847f-c67913d6c579 service neutron] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:49.372240 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063b191c-30b6-40d8-847f-c67913d6c579 service neutron] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:49.372784 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063b191c-30b6-40d8-847f-c67913d6c579 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:49.385669 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063b191c-30b6-40d8-847f-c67913d6c579 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:25:49.388293 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063b191c-30b6-40d8-847f-c67913d6c579 service neutron] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:25:49.402478 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1372/2769] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:25:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 09:25:49.708586 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0458563d-d5b7-4b87-acda-580c6f708143 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:25:49.710318 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-0458563d-d5b7-4b87-acda-580c6f708143 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:25:49.722537 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0458563d-d5b7-4b87-acda-580c6f708143 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:25:49.727052 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5ed43e-a9e4-4218-8cb4-fcace6b3c620 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:25:49.727962 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5ed43e-a9e4-4218-8cb4-fcace6b3c620 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:49.728245 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5ed43e-a9e4-4218-8cb4-fcace6b3c620 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:49.728524 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5ed43e-a9e4-4218-8cb4-fcace6b3c620 admin admin] PATH_INFO: `/v3/projects/de9f3a5158164f1abc0b6090e83e783c` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:49.729681 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5ed43e-a9e4-4218-8cb4-fcace6b3c620 admin admin] RBAC: Authorizing `identity:delete_project(project_id=de9f3a5158164f1abc0b6090e83e783c)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:25:49.731652 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5ed43e-a9e4-4218-8cb4-fcace6b3c620 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:25:49.779500 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-ca5ed43e-a9e4-4218-8cb4-fcace6b3c620 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'de9f3a5158164f1abc0b6090e83e783c', 'request_id': 'req-ca5ed43e-a9e4-4218-8cb4-fcace6b3c620'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:25:49.785636 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-ca5ed43e-a9e4-4218-8cb4-fcace6b3c620 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'de9f3a5158164f1abc0b6090e83e783c', 'request_id': 'req-ca5ed43e-a9e4-4218-8cb4-fcace6b3c620'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:25:49.791407 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-ca5ed43e-a9e4-4218-8cb4-fcace6b3c620 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'de9f3a5158164f1abc0b6090e83e783c', 'request_id': 'req-ca5ed43e-a9e4-4218-8cb4-fcace6b3c620'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:25:49.805236 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-ca5ed43e-a9e4-4218-8cb4-fcace6b3c620 admin admin] The token cache is being invalidate because project de9f3a5158164f1abc0b6090e83e783c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80382) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 09:25:49.805531 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-ca5ed43e-a9e4-4218-8cb4-fcace6b3c620 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:25:49.806149 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-ca5ed43e-a9e4-4218-8cb4-fcace6b3c620 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:25:49.807782 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1398/2770] 10.222.0.26 () {58 vars in 1232 bytes} [Thu Jul 14 09:25:49 2022] DELETE /identity/v3/projects/de9f3a5158164f1abc0b6090e83e783c => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:25:49.812116 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063b191c-30b6-40d8-847f-c67913d6c579 service neutron] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:25:49.821860 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063b191c-30b6-40d8-847f-c67913d6c579 service neutron] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:25:49.880393 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e612fe7-b15c-4e34-8ada-46419a8cde53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:25:49.881270 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e612fe7-b15c-4e34-8ada-46419a8cde53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:49.881556 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e612fe7-b15c-4e34-8ada-46419a8cde53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:49.881839 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e612fe7-b15c-4e34-8ada-46419a8cde53 admin admin] PATH_INFO: `/v3/users/12333f1566524405a38401efb77c124f` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:49.883029 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e612fe7-b15c-4e34-8ada-46419a8cde53 admin admin] RBAC: Authorizing `identity:delete_user(user_id=12333f1566524405a38401efb77c124f)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:25:49.885099 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e612fe7-b15c-4e34-8ada-46419a8cde53 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:25:49.941110 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-3e612fe7-b15c-4e34-8ada-46419a8cde53 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '12333f1566524405a38401efb77c124f'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:25:49.950314 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-3e612fe7-b15c-4e34-8ada-46419a8cde53 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '12333f1566524405a38401efb77c124f'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:25:49.951373 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-3e612fe7-b15c-4e34-8ada-46419a8cde53 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '12333f1566524405a38401efb77c124f'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:25:49.951994 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-3e612fe7-b15c-4e34-8ada-46419a8cde53 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '12333f1566524405a38401efb77c124f'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:25:49.961298 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-3e612fe7-b15c-4e34-8ada-46419a8cde53 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '12333f1566524405a38401efb77c124f'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:25:49.977400 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:25:49.982073 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1373/2771] 10.222.0.26 () {58 vars in 1223 bytes} [Thu Jul 14 09:25:49 2022] DELETE /identity/v3/users/12333f1566524405a38401efb77c124f => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:25:50.342171 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5ed43e-a9e4-4218-8cb4-fcace6b3c620 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:25:50.343863 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-ca5ed43e-a9e4-4218-8cb4-fcace6b3c620 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:25:50.356110 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e612fe7-b15c-4e34-8ada-46419a8cde53 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:25:50.357944 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-3e612fe7-b15c-4e34-8ada-46419a8cde53 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:25:50.368892 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:INSERT=2,keystone:SELECT=91,keystone:DELETE=7 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:25:50.409129 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5ed43e-a9e4-4218-8cb4-fcace6b3c620 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:25:50.439169 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e612fe7-b15c-4e34-8ada-46419a8cde53 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:25:50.443924 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8b8d9a-4f6a-49f4-98a0-f221a491e11d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:25:50.444863 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8b8d9a-4f6a-49f4-98a0-f221a491e11d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:50.445295 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8b8d9a-4f6a-49f4-98a0-f221a491e11d admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:50.445615 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8b8d9a-4f6a-49f4-98a0-f221a491e11d admin admin] PATH_INFO: `/v3/projects/f9491a680e4941428a0cd9bc6b1e4e40` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:50.447273 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8b8d9a-4f6a-49f4-98a0-f221a491e11d admin admin] RBAC: Authorizing `identity:delete_project(project_id=f9491a680e4941428a0cd9bc6b1e4e40)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:25:50.449735 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8b8d9a-4f6a-49f4-98a0-f221a491e11d admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:25:50.460282 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8e7ff5-545a-4c89-b8a3-e74beb187847 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '952911c492444ffea4b211a24b564117', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:25:50.460948 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8e7ff5-545a-4c89-b8a3-e74beb187847 service neutron] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:50.461357 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8e7ff5-545a-4c89-b8a3-e74beb187847 service neutron] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:50.461675 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8e7ff5-545a-4c89-b8a3-e74beb187847 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:50.463524 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-dc8e7ff5-545a-4c89-b8a3-e74beb187847 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:25:50.501169 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-4a8b8d9a-4f6a-49f4-98a0-f221a491e11d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f9491a680e4941428a0cd9bc6b1e4e40', 'request_id': 'req-4a8b8d9a-4f6a-49f4-98a0-f221a491e11d'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:25:50.507650 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-4a8b8d9a-4f6a-49f4-98a0-f221a491e11d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f9491a680e4941428a0cd9bc6b1e4e40', 'request_id': 'req-4a8b8d9a-4f6a-49f4-98a0-f221a491e11d'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:25:50.514634 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-4a8b8d9a-4f6a-49f4-98a0-f221a491e11d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f9491a680e4941428a0cd9bc6b1e4e40', 'request_id': 'req-4a8b8d9a-4f6a-49f4-98a0-f221a491e11d'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:25:50.527580 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-4a8b8d9a-4f6a-49f4-98a0-f221a491e11d admin admin] The token cache is being invalidate because project f9491a680e4941428a0cd9bc6b1e4e40 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80381) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 09:25:50.527870 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-4a8b8d9a-4f6a-49f4-98a0-f221a491e11d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:25:50.528499 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-4a8b8d9a-4f6a-49f4-98a0-f221a491e11d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:25:50.529022 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8e7ff5-545a-4c89-b8a3-e74beb187847 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:25:50.530297 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1374/2772] 10.222.0.26 () {58 vars in 1232 bytes} [Thu Jul 14 09:25:50 2022] DELETE /identity/v3/projects/f9491a680e4941428a0cd9bc6b1e4e40 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:25:50.530627 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8e7ff5-545a-4c89-b8a3-e74beb187847 service neutron] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:25:50.535155 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9c707b-bedd-4f35-a14d-d13a145384b2 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:50.535478 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9c707b-bedd-4f35-a14d-d13a145384b2 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:50.535758 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9c707b-bedd-4f35-a14d-d13a145384b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:50.541228 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1399/2773] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:25:50 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:25:50.556042 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-bd9c707b-bedd-4f35-a14d-d13a145384b2 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:25:50.557344 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-bd9c707b-bedd-4f35-a14d-d13a145384b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:25:50.624113 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1375/2774] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:25:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 09:25:50.633404 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf4eba8-a3e0-4f02-8322-a45a3d6d141e None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:50.633695 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf4eba8-a3e0-4f02-8322-a45a3d6d141e None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:50.634832 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf4eba8-a3e0-4f02-8322-a45a3d6d141e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:50.666497 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-dbf4eba8-a3e0-4f02-8322-a45a3d6d141e None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:25:50.667680 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-dbf4eba8-a3e0-4f02-8322-a45a3d6d141e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:25:50.677331 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1400/2775] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:25:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:25:50.690427 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9c707b-bedd-4f35-a14d-d13a145384b2 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:25:50.699920 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9c707b-bedd-4f35-a14d-d13a145384b2 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:25:50.705421 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af8dbc5-9880-4719-8f32-4c860238ddc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:25:50.706181 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af8dbc5-9880-4719-8f32-4c860238ddc4 admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:50.706537 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af8dbc5-9880-4719-8f32-4c860238ddc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:50.706887 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af8dbc5-9880-4719-8f32-4c860238ddc4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:50.707865 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af8dbc5-9880-4719-8f32-4c860238ddc4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:25:50.708737 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af8dbc5-9880-4719-8f32-4c860238ddc4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:25:50.710009 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af8dbc5-9880-4719-8f32-4c860238ddc4 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:25:50.724574 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1376/2776] 10.222.0.26 () {58 vars in 1152 bytes} [Thu Jul 14 09:25:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 09:25:50.728712 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf4eba8-a3e0-4f02-8322-a45a3d6d141e None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:25:50.750179 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf4eba8-a3e0-4f02-8322-a45a3d6d141e None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:25:50.755061 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c91be10-ef8b-4857-a6d4-6fa4bf6511b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:25:50.755988 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c91be10-ef8b-4857-a6d4-6fa4bf6511b3 admin admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:50.756275 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c91be10-ef8b-4857-a6d4-6fa4bf6511b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:50.756550 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c91be10-ef8b-4857-a6d4-6fa4bf6511b3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:50.757247 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c91be10-ef8b-4857-a6d4-6fa4bf6511b3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:25:50.759143 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c91be10-ef8b-4857-a6d4-6fa4bf6511b3 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:25:50.788710 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1401/2777] 10.222.0.26 () {60 vars in 1152 bytes} [Thu Jul 14 09:25:50 2022] POST /identity/v3/projects => generated 383 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 09:25:50.793173 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af8dbc5-9880-4719-8f32-4c860238ddc4 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:25:50.804421 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af8dbc5-9880-4719-8f32-4c860238ddc4 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:25:50.824970 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:25:50.888614 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5182398-a868-483f-b598-c24bd8ed355f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:25:50.889422 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5182398-a868-483f-b598-c24bd8ed355f admin admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:50.889804 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5182398-a868-483f-b598-c24bd8ed355f admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:50.890152 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5182398-a868-483f-b598-c24bd8ed355f admin admin] PATH_INFO: `/v3/users` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:50.890982 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5182398-a868-483f-b598-c24bd8ed355f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:25:50.893159 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5182398-a868-483f-b598-c24bd8ed355f admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:25:50.917025 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1377/2778] 10.222.0.26 () {60 vars in 1143 bytes} [Thu Jul 14 09:25:50 2022] POST /identity/v3/users => generated 292 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 09:25:50.921733 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c91be10-ef8b-4857-a6d4-6fa4bf6511b3 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:25:50.934832 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c91be10-ef8b-4857-a6d4-6fa4bf6511b3 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:25:50.940211 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edbcc27-89ad-4afa-8db8-09efee583461 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:25:50.940903 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edbcc27-89ad-4afa-8db8-09efee583461 admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:50.941345 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edbcc27-89ad-4afa-8db8-09efee583461 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:50.941619 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edbcc27-89ad-4afa-8db8-09efee583461 admin admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:50.942369 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edbcc27-89ad-4afa-8db8-09efee583461 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:25:50.944085 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edbcc27-89ad-4afa-8db8-09efee583461 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:25:50.956094 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1402/2779] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jul 14 09:25:50 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 09:25:50.960317 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5182398-a868-483f-b598-c24bd8ed355f admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:25:50.972291 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5182398-a868-483f-b598-c24bd8ed355f admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:25:50.985991 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981902c1-ade5-4141-93e9-82fc1ca29a10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:25:50.986783 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981902c1-ade5-4141-93e9-82fc1ca29a10 admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:50.987163 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981902c1-ade5-4141-93e9-82fc1ca29a10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:50.987568 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981902c1-ade5-4141-93e9-82fc1ca29a10 admin admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:50.988632 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981902c1-ade5-4141-93e9-82fc1ca29a10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:25:50.990628 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981902c1-ade5-4141-93e9-82fc1ca29a10 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:25:51.007221 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1378/2780] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jul 14 09:25:50 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 09:25:51.009017 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edbcc27-89ad-4afa-8db8-09efee583461 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:25:51.021564 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edbcc27-89ad-4afa-8db8-09efee583461 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:25:51.031811 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f9cda9-cda9-43d8-b570-10debb87b91a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:25:51.033506 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f9cda9-cda9-43d8-b570-10debb87b91a admin admin] REQUEST_METHOD: `PUT` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:51.033894 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f9cda9-cda9-43d8-b570-10debb87b91a admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:51.034271 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f9cda9-cda9-43d8-b570-10debb87b91a admin admin] PATH_INFO: `/v3/projects/ca60486c5ce64b5c8733f8c910dfb60f/users/243fbc3199df4bdebfdca0ba2151e572/roles/d1691322ce6a4edab7ce722edf0f2988` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:51.076813 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f9cda9-cda9-43d8-b570-10debb87b91a admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca60486c5ce64b5c8733f8c910dfb60f, user_id=243fbc3199df4bdebfdca0ba2151e572, role_id=d1691322ce6a4edab7ce722edf0f2988)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:25:51.079911 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f9cda9-cda9-43d8-b570-10debb87b91a admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:25:51.099850 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1403/2781] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jul 14 09:25:51 2022] PUT /identity/v3/projects/ca60486c5ce64b5c8733f8c910dfb60f/users/243fbc3199df4bdebfdca0ba2151e572/roles/d1691322ce6a4edab7ce722edf0f2988 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 09:25:51.111495 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10ca173-91d1-49e4-9225-bfa18d3d460f None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:51.111903 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10ca173-91d1-49e4-9225-bfa18d3d460f None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:51.112283 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10ca173-91d1-49e4-9225-bfa18d3d460f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:51.142118 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-d10ca173-91d1-49e4-9225-bfa18d3d460f None None] MFA Rules not processed for user `243fbc3199df4bdebfdca0ba2151e572`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:25:51.143366 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-d10ca173-91d1-49e4-9225-bfa18d3d460f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:25:51.211748 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1379/2782] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:25:51 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:25:51.216099 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f9cda9-cda9-43d8-b570-10debb87b91a admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:25:51.241444 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:25:51.281179 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f9cda9-cda9-43d8-b570-10debb87b91a admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:25:51.285850 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fa6f13-088f-4557-a647-22ba54043ce8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:25:51.286510 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fa6f13-088f-4557-a647-22ba54043ce8 admin admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:51.286785 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fa6f13-088f-4557-a647-22ba54043ce8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:51.287078 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fa6f13-088f-4557-a647-22ba54043ce8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:51.287763 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fa6f13-088f-4557-a647-22ba54043ce8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:25:51.289634 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fa6f13-088f-4557-a647-22ba54043ce8 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:25:51.320445 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1404/2783] 10.222.0.26 () {60 vars in 1152 bytes} [Thu Jul 14 09:25:51 2022] POST /identity/v3/projects => generated 385 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 09:25:51.321832 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10ca173-91d1-49e4-9225-bfa18d3d460f None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:25:51.336525 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10ca173-91d1-49e4-9225-bfa18d3d460f None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:25:51.397122 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec05a131-d741-438e-a238-1ee3c1069437 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:25:51.398147 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec05a131-d741-438e-a238-1ee3c1069437 admin admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:51.398617 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec05a131-d741-438e-a238-1ee3c1069437 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:51.398895 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec05a131-d741-438e-a238-1ee3c1069437 admin admin] PATH_INFO: `/v3/users` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:51.399630 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec05a131-d741-438e-a238-1ee3c1069437 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:25:51.401557 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec05a131-d741-438e-a238-1ee3c1069437 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:25:51.424169 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1380/2784] 10.222.0.26 () {60 vars in 1143 bytes} [Thu Jul 14 09:25:51 2022] POST /identity/v3/users => generated 299 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 09:25:51.428179 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fa6f13-088f-4557-a647-22ba54043ce8 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:25:51.438700 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fa6f13-088f-4557-a647-22ba54043ce8 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:25:51.443827 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8908b3ae-2caa-41e1-baad-070aa2cdced6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:25:51.444722 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8908b3ae-2caa-41e1-baad-070aa2cdced6 admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:51.445094 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8908b3ae-2caa-41e1-baad-070aa2cdced6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:51.445567 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8908b3ae-2caa-41e1-baad-070aa2cdced6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:51.446735 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8908b3ae-2caa-41e1-baad-070aa2cdced6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:25:51.448687 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8908b3ae-2caa-41e1-baad-070aa2cdced6 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:25:51.461364 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1405/2785] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jul 14 09:25:51 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 09:25:51.465759 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec05a131-d741-438e-a238-1ee3c1069437 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:25:51.476459 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec05a131-d741-438e-a238-1ee3c1069437 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:25:51.481709 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d727b5-7686-4615-8d05-3e98ecf7369e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:25:51.482605 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d727b5-7686-4615-8d05-3e98ecf7369e admin admin] REQUEST_METHOD: `PUT` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:51.482893 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d727b5-7686-4615-8d05-3e98ecf7369e admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:51.483226 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d727b5-7686-4615-8d05-3e98ecf7369e admin admin] PATH_INFO: `/v3/projects/52c17d708a614a57ad6d25d229724235/users/483a1217da124130846642db5e95cfd1/roles/7b549d58d1724395b343609d895c80d0` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:51.497099 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:25:51.500450 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d727b5-7686-4615-8d05-3e98ecf7369e admin admin] RBAC: Authorizing `identity:create_grant(project_id=52c17d708a614a57ad6d25d229724235, user_id=483a1217da124130846642db5e95cfd1, role_id=7b549d58d1724395b343609d895c80d0)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:25:51.503907 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d727b5-7686-4615-8d05-3e98ecf7369e admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:25:51.518397 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1381/2786] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jul 14 09:25:51 2022] PUT /identity/v3/projects/52c17d708a614a57ad6d25d229724235/users/483a1217da124130846642db5e95cfd1/roles/7b549d58d1724395b343609d895c80d0 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 09:25:51.522876 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8908b3ae-2caa-41e1-baad-070aa2cdced6 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:25:51.533228 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8908b3ae-2caa-41e1-baad-070aa2cdced6 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:25:51.591545 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d8ef37-a143-4e03-9c5b-658861e466bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:25:51.592516 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d8ef37-a143-4e03-9c5b-658861e466bc admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:51.592796 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d8ef37-a143-4e03-9c5b-658861e466bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:51.593068 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d8ef37-a143-4e03-9c5b-658861e466bc admin admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:51.593852 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d8ef37-a143-4e03-9c5b-658861e466bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:25:51.595483 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d8ef37-a143-4e03-9c5b-658861e466bc admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:25:51.605418 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1406/2787] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jul 14 09:25:51 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 09:25:51.610071 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d727b5-7686-4615-8d05-3e98ecf7369e admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:25:51.622224 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d727b5-7686-4615-8d05-3e98ecf7369e admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:25:51.627232 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3a9528-1746-4c3e-98da-430acbd1ed2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:25:51.628143 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3a9528-1746-4c3e-98da-430acbd1ed2b admin admin] REQUEST_METHOD: `PUT` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:51.628421 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3a9528-1746-4c3e-98da-430acbd1ed2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:51.628689 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3a9528-1746-4c3e-98da-430acbd1ed2b admin admin] PATH_INFO: `/v3/projects/52c17d708a614a57ad6d25d229724235/users/483a1217da124130846642db5e95cfd1/roles/7b549d58d1724395b343609d895c80d0` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:51.630705 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3a9528-1746-4c3e-98da-430acbd1ed2b admin admin] RBAC: Authorizing `identity:create_grant(project_id=52c17d708a614a57ad6d25d229724235, user_id=483a1217da124130846642db5e95cfd1, role_id=7b549d58d1724395b343609d895c80d0)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:25:51.633913 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3a9528-1746-4c3e-98da-430acbd1ed2b admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:25:51.644777 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1382/2788] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jul 14 09:25:51 2022] PUT /identity/v3/projects/52c17d708a614a57ad6d25d229724235/users/483a1217da124130846642db5e95cfd1/roles/7b549d58d1724395b343609d895c80d0 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:25:51.655326 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989a5019-dc7a-45cc-8981-d418d34b43b5 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:51.655587 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989a5019-dc7a-45cc-8981-d418d34b43b5 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:51.655896 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989a5019-dc7a-45cc-8981-d418d34b43b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:51.682162 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-989a5019-dc7a-45cc-8981-d418d34b43b5 None None] MFA Rules not processed for user `483a1217da124130846642db5e95cfd1`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:25:51.683427 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-989a5019-dc7a-45cc-8981-d418d34b43b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:25:51.753423 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1407/2789] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:25:51 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:25:51.764439 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3a9528-1746-4c3e-98da-430acbd1ed2b admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:25:51.767012 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-4c3a9528-1746-4c3e-98da-430acbd1ed2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:25:51.833189 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3a9528-1746-4c3e-98da-430acbd1ed2b admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:25:51.837090 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ac734b-4113-45b3-97ec-326918c95346 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e76f1952a5b146808844b9f2b7395035', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:25:51.837831 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ac734b-4113-45b3-97ec-326918c95346 service cinder] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:51.838141 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ac734b-4113-45b3-97ec-326918c95346 service cinder] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:51.838670 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ac734b-4113-45b3-97ec-326918c95346 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:51.848777 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ac734b-4113-45b3-97ec-326918c95346 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:25:51.850345 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ac734b-4113-45b3-97ec-326918c95346 service cinder] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:25:51.860708 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1383/2790] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:25:51 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:25:52.435041 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989a5019-dc7a-45cc-8981-d418d34b43b5 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:25:52.455002 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:25:52.499315 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989a5019-dc7a-45cc-8981-d418d34b43b5 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:25:52.504316 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00407fc-1b4c-4abd-87df-dcc71e66ef81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:25:52.505269 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00407fc-1b4c-4abd-87df-dcc71e66ef81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:52.505649 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00407fc-1b4c-4abd-87df-dcc71e66ef81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:52.505994 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00407fc-1b4c-4abd-87df-dcc71e66ef81 admin admin] PATH_INFO: `/v3/users/243fbc3199df4bdebfdca0ba2151e572` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:52.507577 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00407fc-1b4c-4abd-87df-dcc71e66ef81 admin admin] RBAC: Authorizing `identity:delete_user(user_id=243fbc3199df4bdebfdca0ba2151e572)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:25:52.510548 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00407fc-1b4c-4abd-87df-dcc71e66ef81 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:25:52.573536 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-b00407fc-1b4c-4abd-87df-dcc71e66ef81 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '243fbc3199df4bdebfdca0ba2151e572'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:25:52.574265 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-b00407fc-1b4c-4abd-87df-dcc71e66ef81 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '243fbc3199df4bdebfdca0ba2151e572'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:25:52.587330 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-b00407fc-1b4c-4abd-87df-dcc71e66ef81 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '243fbc3199df4bdebfdca0ba2151e572'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:25:52.596335 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-b00407fc-1b4c-4abd-87df-dcc71e66ef81 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '243fbc3199df4bdebfdca0ba2151e572'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:25:52.596975 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-b00407fc-1b4c-4abd-87df-dcc71e66ef81 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '243fbc3199df4bdebfdca0ba2151e572'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:25:52.612419 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1408/2791] 10.222.0.26 () {58 vars in 1223 bytes} [Thu Jul 14 09:25:52 2022] DELETE /identity/v3/users/243fbc3199df4bdebfdca0ba2151e572 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:25:52.616767 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a5e356-88f8-4f63-af38-f7bb75208884 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:52.617338 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a5e356-88f8-4f63-af38-f7bb75208884 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:52.617445 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a5e356-88f8-4f63-af38-f7bb75208884 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:52.635850 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-28a5e356-88f8-4f63-af38-f7bb75208884 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:25:52.637074 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-28a5e356-88f8-4f63-af38-f7bb75208884 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:25:52.697071 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1384/2792] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:25:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:25:52.707219 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00407fc-1b4c-4abd-87df-dcc71e66ef81 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:25:52.708838 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-b00407fc-1b4c-4abd-87df-dcc71e66ef81 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:25:52.770866 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00407fc-1b4c-4abd-87df-dcc71e66ef81 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:25:52.774637 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fd3ce0-336d-48c7-bcd1-96545cb311a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '952911c492444ffea4b211a24b564117', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:25:52.775320 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fd3ce0-336d-48c7-bcd1-96545cb311a1 service neutron] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:52.775602 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fd3ce0-336d-48c7-bcd1-96545cb311a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:52.775873 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fd3ce0-336d-48c7-bcd1-96545cb311a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:52.785677 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fd3ce0-336d-48c7-bcd1-96545cb311a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:25:52.787140 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fd3ce0-336d-48c7-bcd1-96545cb311a1 service neutron] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:25:52.797180 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1409/2793] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:25:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:25:53.077708 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a5e356-88f8-4f63-af38-f7bb75208884 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:25:53.079368 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-28a5e356-88f8-4f63-af38-f7bb75208884 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:25:53.117431 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a5e356-88f8-4f63-af38-f7bb75208884 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:25:53.125555 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5d4e51-12f6-459f-af8a-72c8fe58c679 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:25:53.126844 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5d4e51-12f6-459f-af8a-72c8fe58c679 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:53.126844 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5d4e51-12f6-459f-af8a-72c8fe58c679 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:53.127277 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5d4e51-12f6-459f-af8a-72c8fe58c679 admin admin] PATH_INFO: `/v3/projects/ca60486c5ce64b5c8733f8c910dfb60f` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:53.129326 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5d4e51-12f6-459f-af8a-72c8fe58c679 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ca60486c5ce64b5c8733f8c910dfb60f)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:25:53.131457 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5d4e51-12f6-459f-af8a-72c8fe58c679 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:25:53.188814 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-1d5d4e51-12f6-459f-af8a-72c8fe58c679 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca60486c5ce64b5c8733f8c910dfb60f', 'request_id': 'req-1d5d4e51-12f6-459f-af8a-72c8fe58c679'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:25:53.196501 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-1d5d4e51-12f6-459f-af8a-72c8fe58c679 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca60486c5ce64b5c8733f8c910dfb60f', 'request_id': 'req-1d5d4e51-12f6-459f-af8a-72c8fe58c679'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:25:53.209819 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-1d5d4e51-12f6-459f-af8a-72c8fe58c679 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca60486c5ce64b5c8733f8c910dfb60f', 'request_id': 'req-1d5d4e51-12f6-459f-af8a-72c8fe58c679'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:25:53.240309 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-1d5d4e51-12f6-459f-af8a-72c8fe58c679 admin admin] The token cache is being invalidate because project ca60486c5ce64b5c8733f8c910dfb60f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80381) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 09:25:53.240686 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-1d5d4e51-12f6-459f-af8a-72c8fe58c679 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:25:53.241337 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-1d5d4e51-12f6-459f-af8a-72c8fe58c679 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:25:53.243015 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1385/2794] 10.222.0.26 () {58 vars in 1232 bytes} [Thu Jul 14 09:25:53 2022] DELETE /identity/v3/projects/ca60486c5ce64b5c8733f8c910dfb60f => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:25:53.247395 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fd3ce0-336d-48c7-bcd1-96545cb311a1 service neutron] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:25:53.258777 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fd3ce0-336d-48c7-bcd1-96545cb311a1 service neutron] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:25:53.302621 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:25:53.323497 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f038b5-10d5-4f5b-9992-1e810aa0b605 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:25:53.324515 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f038b5-10d5-4f5b-9992-1e810aa0b605 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:53.324955 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f038b5-10d5-4f5b-9992-1e810aa0b605 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:53.325063 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f038b5-10d5-4f5b-9992-1e810aa0b605 admin admin] PATH_INFO: `/v3/users/483a1217da124130846642db5e95cfd1` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:53.326256 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f038b5-10d5-4f5b-9992-1e810aa0b605 admin admin] RBAC: Authorizing `identity:delete_user(user_id=483a1217da124130846642db5e95cfd1)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:25:53.328287 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f038b5-10d5-4f5b-9992-1e810aa0b605 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:25:53.381173 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-79f038b5-10d5-4f5b-9992-1e810aa0b605 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '483a1217da124130846642db5e95cfd1'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:25:53.381854 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-79f038b5-10d5-4f5b-9992-1e810aa0b605 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '483a1217da124130846642db5e95cfd1'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:25:53.396706 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-79f038b5-10d5-4f5b-9992-1e810aa0b605 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '483a1217da124130846642db5e95cfd1'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:25:53.406167 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-79f038b5-10d5-4f5b-9992-1e810aa0b605 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '483a1217da124130846642db5e95cfd1'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:25:53.406908 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-79f038b5-10d5-4f5b-9992-1e810aa0b605 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '483a1217da124130846642db5e95cfd1'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:25:53.427363 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1410/2795] 10.222.0.26 () {58 vars in 1223 bytes} [Thu Jul 14 09:25:53 2022] DELETE /identity/v3/users/483a1217da124130846642db5e95cfd1 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:25:53.815062 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5d4e51-12f6-459f-af8a-72c8fe58c679 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:25:53.816688 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-1d5d4e51-12f6-459f-af8a-72c8fe58c679 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:25:53.826872 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:25:53.831519 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5d4e51-12f6-459f-af8a-72c8fe58c679 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:25:53.894920 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edbcf60-765d-46a0-a8e0-a3632c2532ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:25:53.896021 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edbcf60-765d-46a0-a8e0-a3632c2532ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:53.896304 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edbcf60-765d-46a0-a8e0-a3632c2532ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:53.896575 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edbcf60-765d-46a0-a8e0-a3632c2532ec admin admin] PATH_INFO: `/v3/projects/52c17d708a614a57ad6d25d229724235` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:53.897753 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edbcf60-765d-46a0-a8e0-a3632c2532ec admin admin] RBAC: Authorizing `identity:delete_project(project_id=52c17d708a614a57ad6d25d229724235)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:25:53.899699 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edbcf60-765d-46a0-a8e0-a3632c2532ec admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:25:53.948327 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-6edbcf60-765d-46a0-a8e0-a3632c2532ec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '52c17d708a614a57ad6d25d229724235', 'request_id': 'req-6edbcf60-765d-46a0-a8e0-a3632c2532ec'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:25:53.954582 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-6edbcf60-765d-46a0-a8e0-a3632c2532ec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '52c17d708a614a57ad6d25d229724235', 'request_id': 'req-6edbcf60-765d-46a0-a8e0-a3632c2532ec'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:25:53.960241 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-6edbcf60-765d-46a0-a8e0-a3632c2532ec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '52c17d708a614a57ad6d25d229724235', 'request_id': 'req-6edbcf60-765d-46a0-a8e0-a3632c2532ec'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:25:53.973772 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-6edbcf60-765d-46a0-a8e0-a3632c2532ec admin admin] The token cache is being invalidate because project 52c17d708a614a57ad6d25d229724235 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80381) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 09:25:53.974096 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-6edbcf60-765d-46a0-a8e0-a3632c2532ec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:25:53.974785 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-6edbcf60-765d-46a0-a8e0-a3632c2532ec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:25:53.976403 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1386/2796] 10.222.0.26 () {58 vars in 1232 bytes} [Thu Jul 14 09:25:53 2022] DELETE /identity/v3/projects/52c17d708a614a57ad6d25d229724235 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 09:25:53.981480 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f521f0ad-8966-43eb-8dce-314cf6234622 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:53.981975 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f521f0ad-8966-43eb-8dce-314cf6234622 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:53.982088 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f521f0ad-8966-43eb-8dce-314cf6234622 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:54.001071 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-f521f0ad-8966-43eb-8dce-314cf6234622 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:25:54.002449 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-f521f0ad-8966-43eb-8dce-314cf6234622 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:25:54.062867 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1411/2797] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:25:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:25:54.127718 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8034cdf-a10d-4757-862a-14425b2cd7b7 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:54.127989 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8034cdf-a10d-4757-862a-14425b2cd7b7 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:54.128272 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8034cdf-a10d-4757-862a-14425b2cd7b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:54.148894 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-f8034cdf-a10d-4757-862a-14425b2cd7b7 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:25:54.150134 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-f8034cdf-a10d-4757-862a-14425b2cd7b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:25:54.159217 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1387/2798] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:25:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:25:54.163364 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f521f0ad-8966-43eb-8dce-314cf6234622 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:25:54.174063 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f521f0ad-8966-43eb-8dce-314cf6234622 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:25:54.179313 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c27252-ede4-4409-9894-ec2313b05941 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:25:54.179979 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c27252-ede4-4409-9894-ec2313b05941 admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:54.180258 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c27252-ede4-4409-9894-ec2313b05941 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:54.180527 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c27252-ede4-4409-9894-ec2313b05941 admin admin] PATH_INFO: `/v3/domains` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:54.181251 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c27252-ede4-4409-9894-ec2313b05941 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:25:54.181554 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c27252-ede4-4409-9894-ec2313b05941 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:25:54.183138 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c27252-ede4-4409-9894-ec2313b05941 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:25:54.197242 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1412/2799] 10.222.0.26 () {58 vars in 1152 bytes} [Thu Jul 14 09:25:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:25:54.201464 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8034cdf-a10d-4757-862a-14425b2cd7b7 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:25:54.212367 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8034cdf-a10d-4757-862a-14425b2cd7b7 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:25:54.217833 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816669f2-f262-47ef-b9b9-baae8f5b820d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:25:54.218524 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816669f2-f262-47ef-b9b9-baae8f5b820d admin admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:54.218945 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816669f2-f262-47ef-b9b9-baae8f5b820d admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:54.219640 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816669f2-f262-47ef-b9b9-baae8f5b820d admin admin] PATH_INFO: `/v3/projects` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:54.219973 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816669f2-f262-47ef-b9b9-baae8f5b820d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:25:54.221924 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816669f2-f262-47ef-b9b9-baae8f5b820d admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:25:54.269146 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1388/2800] 10.222.0.26 () {60 vars in 1152 bytes} [Thu Jul 14 09:25:54 2022] POST /identity/v3/projects => generated 373 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 09:25:54.273626 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c27252-ede4-4409-9894-ec2313b05941 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:25:54.298139 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c27252-ede4-4409-9894-ec2313b05941 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:25:54.353256 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:25:54.367925 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a89977-82e6-44d8-a6a5-957ec22f29c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:25:54.368659 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a89977-82e6-44d8-a6a5-957ec22f29c9 admin admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:54.368945 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a89977-82e6-44d8-a6a5-957ec22f29c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:54.369224 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a89977-82e6-44d8-a6a5-957ec22f29c9 admin admin] PATH_INFO: `/v3/users` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:54.370029 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a89977-82e6-44d8-a6a5-957ec22f29c9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:25:54.371949 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a89977-82e6-44d8-a6a5-957ec22f29c9 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:25:54.391440 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1413/2801] 10.222.0.26 () {60 vars in 1143 bytes} [Thu Jul 14 09:25:54 2022] POST /identity/v3/users => generated 287 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 09:25:54.396383 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816669f2-f262-47ef-b9b9-baae8f5b820d admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:25:54.408418 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816669f2-f262-47ef-b9b9-baae8f5b820d admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:25:54.414059 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058b339e-5f87-4b42-879e-5241fcd40003 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:25:54.414766 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058b339e-5f87-4b42-879e-5241fcd40003 admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:54.415050 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058b339e-5f87-4b42-879e-5241fcd40003 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:54.415362 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058b339e-5f87-4b42-879e-5241fcd40003 admin admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:54.416116 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058b339e-5f87-4b42-879e-5241fcd40003 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:25:54.417758 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058b339e-5f87-4b42-879e-5241fcd40003 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:25:54.429095 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1389/2802] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jul 14 09:25:54 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 09:25:54.433302 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a89977-82e6-44d8-a6a5-957ec22f29c9 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:25:54.443774 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a89977-82e6-44d8-a6a5-957ec22f29c9 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:25:54.449649 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f76e743-6b38-46c8-bffb-40ba9a1267f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:25:54.450438 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f76e743-6b38-46c8-bffb-40ba9a1267f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:54.450756 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f76e743-6b38-46c8-bffb-40ba9a1267f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:54.451055 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f76e743-6b38-46c8-bffb-40ba9a1267f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:54.451940 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f76e743-6b38-46c8-bffb-40ba9a1267f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:25:54.453973 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f76e743-6b38-46c8-bffb-40ba9a1267f4 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:25:54.464579 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1414/2803] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jul 14 09:25:54 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 09:25:54.469025 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058b339e-5f87-4b42-879e-5241fcd40003 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:25:54.480041 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058b339e-5f87-4b42-879e-5241fcd40003 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:25:54.485598 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64724656-2cdb-41ba-92a4-1f6ee08a870e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:25:54.486519 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64724656-2cdb-41ba-92a4-1f6ee08a870e admin admin] REQUEST_METHOD: `PUT` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:54.486790 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64724656-2cdb-41ba-92a4-1f6ee08a870e admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:54.487075 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64724656-2cdb-41ba-92a4-1f6ee08a870e admin admin] PATH_INFO: `/v3/projects/7258b37daf4c42fbb33204e1ae273180/users/d951a7adb8944098afe928e3fa0c0f54/roles/d1691322ce6a4edab7ce722edf0f2988` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:54.504249 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64724656-2cdb-41ba-92a4-1f6ee08a870e admin admin] RBAC: Authorizing `identity:create_grant(project_id=7258b37daf4c42fbb33204e1ae273180, user_id=d951a7adb8944098afe928e3fa0c0f54, role_id=d1691322ce6a4edab7ce722edf0f2988)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:25:54.506935 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64724656-2cdb-41ba-92a4-1f6ee08a870e admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:25:54.524223 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1390/2804] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jul 14 09:25:54 2022] PUT /identity/v3/projects/7258b37daf4c42fbb33204e1ae273180/users/d951a7adb8944098afe928e3fa0c0f54/roles/d1691322ce6a4edab7ce722edf0f2988 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:25:54.535005 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bb455c-f06f-4531-a785-5cf452650a78 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:54.535332 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bb455c-f06f-4531-a785-5cf452650a78 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:54.535612 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bb455c-f06f-4531-a785-5cf452650a78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:54.564438 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-b9bb455c-f06f-4531-a785-5cf452650a78 None None] MFA Rules not processed for user `d951a7adb8944098afe928e3fa0c0f54`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:25:54.565747 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-b9bb455c-f06f-4531-a785-5cf452650a78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:25:54.635791 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1415/2805] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:25:54 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:25:54.640189 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64724656-2cdb-41ba-92a4-1f6ee08a870e admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:25:54.677957 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:25:54.711009 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64724656-2cdb-41ba-92a4-1f6ee08a870e admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:25:54.716150 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a91b626-fb0f-406e-9a96-f0cf4d03cfac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:25:54.716804 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a91b626-fb0f-406e-9a96-f0cf4d03cfac admin admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:54.717093 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a91b626-fb0f-406e-9a96-f0cf4d03cfac admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:54.717355 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a91b626-fb0f-406e-9a96-f0cf4d03cfac admin admin] PATH_INFO: `/v3/projects` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:54.718068 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a91b626-fb0f-406e-9a96-f0cf4d03cfac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:25:54.720251 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a91b626-fb0f-406e-9a96-f0cf4d03cfac admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:25:54.747145 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1391/2806] 10.222.0.26 () {60 vars in 1152 bytes} [Thu Jul 14 09:25:54 2022] POST /identity/v3/projects => generated 373 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 09:25:54.751600 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bb455c-f06f-4531-a785-5cf452650a78 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:25:54.763353 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bb455c-f06f-4531-a785-5cf452650a78 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:25:54.827560 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e368cd-a207-4b86-a60b-6e54ef01609e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:25:54.828242 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e368cd-a207-4b86-a60b-6e54ef01609e admin admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:54.828526 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e368cd-a207-4b86-a60b-6e54ef01609e admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:54.828802 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e368cd-a207-4b86-a60b-6e54ef01609e admin admin] PATH_INFO: `/v3/users` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:54.829556 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e368cd-a207-4b86-a60b-6e54ef01609e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:25:54.831522 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e368cd-a207-4b86-a60b-6e54ef01609e admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:25:54.852301 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1416/2807] 10.222.0.26 () {60 vars in 1143 bytes} [Thu Jul 14 09:25:54 2022] POST /identity/v3/users => generated 293 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 09:25:54.856553 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a91b626-fb0f-406e-9a96-f0cf4d03cfac admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:25:54.867474 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a91b626-fb0f-406e-9a96-f0cf4d03cfac admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:25:54.872600 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba92a2c6-dcfb-4312-9748-1fe05ee5e712 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:25:54.873265 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba92a2c6-dcfb-4312-9748-1fe05ee5e712 admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:54.873574 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba92a2c6-dcfb-4312-9748-1fe05ee5e712 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:54.873849 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba92a2c6-dcfb-4312-9748-1fe05ee5e712 admin admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:54.874624 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba92a2c6-dcfb-4312-9748-1fe05ee5e712 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:25:54.876315 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba92a2c6-dcfb-4312-9748-1fe05ee5e712 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:25:54.887016 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1392/2808] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jul 14 09:25:54 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 09:25:54.891310 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e368cd-a207-4b86-a60b-6e54ef01609e admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:25:54.901834 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e368cd-a207-4b86-a60b-6e54ef01609e admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:25:54.906898 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8846f3d6-33da-49aa-a28e-37ce87607814 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:25:54.907799 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8846f3d6-33da-49aa-a28e-37ce87607814 admin admin] REQUEST_METHOD: `PUT` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:54.908089 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8846f3d6-33da-49aa-a28e-37ce87607814 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:54.908365 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8846f3d6-33da-49aa-a28e-37ce87607814 admin admin] PATH_INFO: `/v3/projects/df2f7222dffb45c0879f0b8711dd99ca/users/ff77e72bcf6e44419419884481b0fb28/roles/7b549d58d1724395b343609d895c80d0` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:54.923500 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8846f3d6-33da-49aa-a28e-37ce87607814 admin admin] RBAC: Authorizing `identity:create_grant(project_id=df2f7222dffb45c0879f0b8711dd99ca, user_id=ff77e72bcf6e44419419884481b0fb28, role_id=7b549d58d1724395b343609d895c80d0)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:25:54.926203 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8846f3d6-33da-49aa-a28e-37ce87607814 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:25:54.942234 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1417/2809] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jul 14 09:25:54 2022] PUT /identity/v3/projects/df2f7222dffb45c0879f0b8711dd99ca/users/ff77e72bcf6e44419419884481b0fb28/roles/7b549d58d1724395b343609d895c80d0 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:25:54.946224 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba92a2c6-dcfb-4312-9748-1fe05ee5e712 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:25:54.956433 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba92a2c6-dcfb-4312-9748-1fe05ee5e712 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:25:55.028952 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2227b90b-5fef-4d5b-b732-7e3a3f67a404 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:25:55.029736 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2227b90b-5fef-4d5b-b732-7e3a3f67a404 admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:55.030019 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2227b90b-5fef-4d5b-b732-7e3a3f67a404 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:55.030293 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2227b90b-5fef-4d5b-b732-7e3a3f67a404 admin admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:55.031030 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2227b90b-5fef-4d5b-b732-7e3a3f67a404 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:25:55.032696 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2227b90b-5fef-4d5b-b732-7e3a3f67a404 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:25:55.043059 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1393/2810] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jul 14 09:25:54 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 09:25:55.047720 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8846f3d6-33da-49aa-a28e-37ce87607814 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:25:55.058272 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8846f3d6-33da-49aa-a28e-37ce87607814 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:25:55.063034 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21903ef-870d-4ffb-9aaa-d3d30ef4001b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:25:55.063947 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21903ef-870d-4ffb-9aaa-d3d30ef4001b admin admin] REQUEST_METHOD: `PUT` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:55.064232 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21903ef-870d-4ffb-9aaa-d3d30ef4001b admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:55.064509 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21903ef-870d-4ffb-9aaa-d3d30ef4001b admin admin] PATH_INFO: `/v3/projects/df2f7222dffb45c0879f0b8711dd99ca/users/ff77e72bcf6e44419419884481b0fb28/roles/7b549d58d1724395b343609d895c80d0` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:55.066533 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21903ef-870d-4ffb-9aaa-d3d30ef4001b admin admin] RBAC: Authorizing `identity:create_grant(project_id=df2f7222dffb45c0879f0b8711dd99ca, user_id=ff77e72bcf6e44419419884481b0fb28, role_id=7b549d58d1724395b343609d895c80d0)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:25:55.069197 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21903ef-870d-4ffb-9aaa-d3d30ef4001b admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:25:55.079663 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1418/2811] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jul 14 09:25:55 2022] PUT /identity/v3/projects/df2f7222dffb45c0879f0b8711dd99ca/users/ff77e72bcf6e44419419884481b0fb28/roles/7b549d58d1724395b343609d895c80d0 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:25:55.090079 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9032d813-3702-43c2-8edc-e7047fe98533 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:55.090368 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9032d813-3702-43c2-8edc-e7047fe98533 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:55.090644 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9032d813-3702-43c2-8edc-e7047fe98533 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:55.117797 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-9032d813-3702-43c2-8edc-e7047fe98533 None None] MFA Rules not processed for user `ff77e72bcf6e44419419884481b0fb28`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:25:55.119010 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-9032d813-3702-43c2-8edc-e7047fe98533 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:25:55.191800 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1394/2812] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:25:55 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 09:25:55.203974 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21903ef-870d-4ffb-9aaa-d3d30ef4001b admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:25:55.206153 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-a21903ef-870d-4ffb-9aaa-d3d30ef4001b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:25:55.217353 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:25:55.268736 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21903ef-870d-4ffb-9aaa-d3d30ef4001b admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:25:55.272829 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97818a3e-32b7-4cf5-80f8-08602c8b9359 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e76f1952a5b146808844b9f2b7395035', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:25:55.273500 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97818a3e-32b7-4cf5-80f8-08602c8b9359 service cinder] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:25:55.273782 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97818a3e-32b7-4cf5-80f8-08602c8b9359 service cinder] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:25:55.274058 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97818a3e-32b7-4cf5-80f8-08602c8b9359 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:25:55.335927 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97818a3e-32b7-4cf5-80f8-08602c8b9359 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:25:55.337365 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97818a3e-32b7-4cf5-80f8-08602c8b9359 service cinder] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:25:55.346898 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1419/2813] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:25:55 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:26:01.073358 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9032d813-3702-43c2-8edc-e7047fe98533 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:26:01.105966 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:26:01.147969 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9032d813-3702-43c2-8edc-e7047fe98533 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:26:01.152946 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b213598-cf96-44d3-bbd4-1da1db538ca6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:26:01.153835 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b213598-cf96-44d3-bbd4-1da1db538ca6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:26:01.154206 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b213598-cf96-44d3-bbd4-1da1db538ca6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:26:01.154503 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b213598-cf96-44d3-bbd4-1da1db538ca6 admin admin] PATH_INFO: `/v3/users/d951a7adb8944098afe928e3fa0c0f54` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:26:01.155813 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b213598-cf96-44d3-bbd4-1da1db538ca6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d951a7adb8944098afe928e3fa0c0f54)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:26:01.157786 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b213598-cf96-44d3-bbd4-1da1db538ca6 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:26:01.209463 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-8b213598-cf96-44d3-bbd4-1da1db538ca6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd951a7adb8944098afe928e3fa0c0f54'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:26:01.217935 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-8b213598-cf96-44d3-bbd4-1da1db538ca6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd951a7adb8944098afe928e3fa0c0f54'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:26:01.218546 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-8b213598-cf96-44d3-bbd4-1da1db538ca6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd951a7adb8944098afe928e3fa0c0f54'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:26:01.219562 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-8b213598-cf96-44d3-bbd4-1da1db538ca6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd951a7adb8944098afe928e3fa0c0f54'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:26:01.226827 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-8b213598-cf96-44d3-bbd4-1da1db538ca6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd951a7adb8944098afe928e3fa0c0f54'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:26:01.244922 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1395/2814] 10.222.0.26 () {58 vars in 1223 bytes} [Thu Jul 14 09:26:01 2022] DELETE /identity/v3/users/d951a7adb8944098afe928e3fa0c0f54 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:26:01.249436 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9676012d-a7b3-437a-93db-8331f6a933c3 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:26:01.249966 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9676012d-a7b3-437a-93db-8331f6a933c3 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:26:01.250112 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9676012d-a7b3-437a-93db-8331f6a933c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:26:01.268060 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-9676012d-a7b3-437a-93db-8331f6a933c3 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:26:01.269271 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-9676012d-a7b3-437a-93db-8331f6a933c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:26:01.334322 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1420/2815] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:26:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 09:26:01.341706 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b213598-cf96-44d3-bbd4-1da1db538ca6 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:26:01.343464 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-8b213598-cf96-44d3-bbd4-1da1db538ca6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:26:01.436968 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b213598-cf96-44d3-bbd4-1da1db538ca6 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:26:01.440951 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437eac55-b8a1-421c-8d68-7387cbb313f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '952911c492444ffea4b211a24b564117', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:26:01.441598 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437eac55-b8a1-421c-8d68-7387cbb313f8 service neutron] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:26:01.441880 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437eac55-b8a1-421c-8d68-7387cbb313f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:26:01.442153 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437eac55-b8a1-421c-8d68-7387cbb313f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:26:01.451591 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437eac55-b8a1-421c-8d68-7387cbb313f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:26:01.453018 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437eac55-b8a1-421c-8d68-7387cbb313f8 service neutron] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:26:01.462735 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1396/2816] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:26:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:26:01.757363 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9676012d-a7b3-437a-93db-8331f6a933c3 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:26:01.758889 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-9676012d-a7b3-437a-93db-8331f6a933c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:26:01.772970 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9676012d-a7b3-437a-93db-8331f6a933c3 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:26:01.792163 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d3eefb-3782-4c9d-840b-f48fec2d3bb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:26:01.793598 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d3eefb-3782-4c9d-840b-f48fec2d3bb5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:26:01.793917 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d3eefb-3782-4c9d-840b-f48fec2d3bb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:26:01.794125 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d3eefb-3782-4c9d-840b-f48fec2d3bb5 admin admin] PATH_INFO: `/v3/projects/7258b37daf4c42fbb33204e1ae273180` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:26:01.796931 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d3eefb-3782-4c9d-840b-f48fec2d3bb5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7258b37daf4c42fbb33204e1ae273180)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:26:01.798949 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d3eefb-3782-4c9d-840b-f48fec2d3bb5 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:26:01.850912 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-b6d3eefb-3782-4c9d-840b-f48fec2d3bb5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7258b37daf4c42fbb33204e1ae273180', 'request_id': 'req-b6d3eefb-3782-4c9d-840b-f48fec2d3bb5'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:26:01.857427 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-b6d3eefb-3782-4c9d-840b-f48fec2d3bb5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7258b37daf4c42fbb33204e1ae273180', 'request_id': 'req-b6d3eefb-3782-4c9d-840b-f48fec2d3bb5'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:26:01.859548 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:26:01.864182 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-b6d3eefb-3782-4c9d-840b-f48fec2d3bb5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7258b37daf4c42fbb33204e1ae273180', 'request_id': 'req-b6d3eefb-3782-4c9d-840b-f48fec2d3bb5'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:26:01.877355 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-b6d3eefb-3782-4c9d-840b-f48fec2d3bb5 admin admin] The token cache is being invalidate because project 7258b37daf4c42fbb33204e1ae273180 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80382) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 09:26:01.877639 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-b6d3eefb-3782-4c9d-840b-f48fec2d3bb5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:26:01.878333 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-b6d3eefb-3782-4c9d-840b-f48fec2d3bb5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:26:01.880027 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1421/2817] 10.222.0.26 () {58 vars in 1232 bytes} [Thu Jul 14 09:26:01 2022] DELETE /identity/v3/projects/7258b37daf4c42fbb33204e1ae273180 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:26:01.883995 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437eac55-b8a1-421c-8d68-7387cbb313f8 service neutron] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:26:01.893971 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437eac55-b8a1-421c-8d68-7387cbb313f8 service neutron] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:26:01.921518 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:26:01.953409 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c474e4f2-4cfa-48e4-b101-4379591d2802 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:26:01.954276 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c474e4f2-4cfa-48e4-b101-4379591d2802 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:26:01.954555 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c474e4f2-4cfa-48e4-b101-4379591d2802 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:26:01.954828 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c474e4f2-4cfa-48e4-b101-4379591d2802 admin admin] PATH_INFO: `/v3/users/ff77e72bcf6e44419419884481b0fb28` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:26:01.956029 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c474e4f2-4cfa-48e4-b101-4379591d2802 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ff77e72bcf6e44419419884481b0fb28)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:26:01.957945 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c474e4f2-4cfa-48e4-b101-4379591d2802 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:26:02.017791 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-c474e4f2-4cfa-48e4-b101-4379591d2802 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ff77e72bcf6e44419419884481b0fb28'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:26:02.026872 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-c474e4f2-4cfa-48e4-b101-4379591d2802 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ff77e72bcf6e44419419884481b0fb28'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:26:02.027634 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-c474e4f2-4cfa-48e4-b101-4379591d2802 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ff77e72bcf6e44419419884481b0fb28'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:26:02.028329 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-c474e4f2-4cfa-48e4-b101-4379591d2802 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ff77e72bcf6e44419419884481b0fb28'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:26:02.035719 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-c474e4f2-4cfa-48e4-b101-4379591d2802 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ff77e72bcf6e44419419884481b0fb28'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:26:02.054935 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1397/2818] 10.222.0.26 () {58 vars in 1223 bytes} [Thu Jul 14 09:26:01 2022] DELETE /identity/v3/users/ff77e72bcf6e44419419884481b0fb28 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:26:02.329433 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d3eefb-3782-4c9d-840b-f48fec2d3bb5 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:26:02.331476 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-b6d3eefb-3782-4c9d-840b-f48fec2d3bb5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:26:02.354936 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d3eefb-3782-4c9d-840b-f48fec2d3bb5 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:26:02.421335 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbea6552-e87e-4131-8917-8e77103822c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:26:02.422115 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbea6552-e87e-4131-8917-8e77103822c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:26:02.422397 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbea6552-e87e-4131-8917-8e77103822c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:26:02.422749 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbea6552-e87e-4131-8917-8e77103822c8 admin admin] PATH_INFO: `/v3/projects/df2f7222dffb45c0879f0b8711dd99ca` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:26:02.424010 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbea6552-e87e-4131-8917-8e77103822c8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=df2f7222dffb45c0879f0b8711dd99ca)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:26:02.425984 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbea6552-e87e-4131-8917-8e77103822c8 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:26:02.472834 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-dbea6552-e87e-4131-8917-8e77103822c8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'df2f7222dffb45c0879f0b8711dd99ca', 'request_id': 'req-dbea6552-e87e-4131-8917-8e77103822c8'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:26:02.479179 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-dbea6552-e87e-4131-8917-8e77103822c8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'df2f7222dffb45c0879f0b8711dd99ca', 'request_id': 'req-dbea6552-e87e-4131-8917-8e77103822c8'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:26:02.488104 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-dbea6552-e87e-4131-8917-8e77103822c8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'df2f7222dffb45c0879f0b8711dd99ca', 'request_id': 'req-dbea6552-e87e-4131-8917-8e77103822c8'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:26:02.503255 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-dbea6552-e87e-4131-8917-8e77103822c8 admin admin] The token cache is being invalidate because project df2f7222dffb45c0879f0b8711dd99ca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80382) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 09:26:02.503548 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-dbea6552-e87e-4131-8917-8e77103822c8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:26:02.504188 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-dbea6552-e87e-4131-8917-8e77103822c8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:26:02.505752 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1422/2819] 10.222.0.26 () {58 vars in 1232 bytes} [Thu Jul 14 09:26:02 2022] DELETE /identity/v3/projects/df2f7222dffb45c0879f0b8711dd99ca => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:26:02.510614 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473c5b54-1efc-4358-ad1b-58462cd4795c None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:26:02.511095 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473c5b54-1efc-4358-ad1b-58462cd4795c None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:26:02.511420 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473c5b54-1efc-4358-ad1b-58462cd4795c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:26:02.530613 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-473c5b54-1efc-4358-ad1b-58462cd4795c None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:26:02.531894 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-473c5b54-1efc-4358-ad1b-58462cd4795c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:26:02.598187 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1398/2820] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:26:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:26:02.606196 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69455af6-594d-43c5-b462-64d73b253349 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:26:02.606479 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69455af6-594d-43c5-b462-64d73b253349 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:26:02.606799 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69455af6-594d-43c5-b462-64d73b253349 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:26:02.625813 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-69455af6-594d-43c5-b462-64d73b253349 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:26:02.627018 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-69455af6-594d-43c5-b462-64d73b253349 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:26:02.635822 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1423/2821] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:26:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:26:02.639986 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473c5b54-1efc-4358-ad1b-58462cd4795c None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:26:02.650285 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473c5b54-1efc-4358-ad1b-58462cd4795c None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:26:02.655320 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b60f9cf-e752-46b1-9156-450e28efda61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:26:02.655966 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b60f9cf-e752-46b1-9156-450e28efda61 admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:26:02.656296 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b60f9cf-e752-46b1-9156-450e28efda61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:26:02.656580 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b60f9cf-e752-46b1-9156-450e28efda61 admin admin] PATH_INFO: `/v3/domains` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:26:02.657307 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b60f9cf-e752-46b1-9156-450e28efda61 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:26:02.657615 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b60f9cf-e752-46b1-9156-450e28efda61 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:26:02.659287 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b60f9cf-e752-46b1-9156-450e28efda61 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:26:02.672360 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1399/2822] 10.222.0.26 () {58 vars in 1152 bytes} [Thu Jul 14 09:26:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:26:02.676700 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69455af6-594d-43c5-b462-64d73b253349 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:26:02.686195 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69455af6-594d-43c5-b462-64d73b253349 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:26:02.691311 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88faeb8e-2423-44a0-8f19-f08f66f5e5db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:26:02.691989 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88faeb8e-2423-44a0-8f19-f08f66f5e5db admin admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:26:02.692274 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88faeb8e-2423-44a0-8f19-f08f66f5e5db admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:26:02.692551 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88faeb8e-2423-44a0-8f19-f08f66f5e5db admin admin] PATH_INFO: `/v3/projects` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:26:02.693295 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88faeb8e-2423-44a0-8f19-f08f66f5e5db admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:26:02.695217 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88faeb8e-2423-44a0-8f19-f08f66f5e5db admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:26:02.730886 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1424/2823] 10.222.0.26 () {60 vars in 1152 bytes} [Thu Jul 14 09:26:02 2022] POST /identity/v3/projects => generated 371 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 09:26:02.735924 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b60f9cf-e752-46b1-9156-450e28efda61 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:26:02.746822 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b60f9cf-e752-46b1-9156-450e28efda61 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:26:02.780654 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:26:02.809137 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4887882-2261-4ca8-a277-a0e1bd69b626 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:26:02.809805 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4887882-2261-4ca8-a277-a0e1bd69b626 admin admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:26:02.810088 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4887882-2261-4ca8-a277-a0e1bd69b626 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:26:02.810363 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4887882-2261-4ca8-a277-a0e1bd69b626 admin admin] PATH_INFO: `/v3/users` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:26:02.811049 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4887882-2261-4ca8-a277-a0e1bd69b626 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:26:02.813110 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4887882-2261-4ca8-a277-a0e1bd69b626 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:26:02.835420 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1400/2824] 10.222.0.26 () {60 vars in 1143 bytes} [Thu Jul 14 09:26:02 2022] POST /identity/v3/users => generated 286 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 09:26:02.839427 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88faeb8e-2423-44a0-8f19-f08f66f5e5db admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:26:02.850432 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88faeb8e-2423-44a0-8f19-f08f66f5e5db admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:26:02.855488 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bdead4-f8f5-4953-a138-45eedd301d4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:26:02.856219 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bdead4-f8f5-4953-a138-45eedd301d4d admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:26:02.856569 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bdead4-f8f5-4953-a138-45eedd301d4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:26:02.857056 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bdead4-f8f5-4953-a138-45eedd301d4d admin admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:26:02.857784 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bdead4-f8f5-4953-a138-45eedd301d4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:26:02.859501 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bdead4-f8f5-4953-a138-45eedd301d4d admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:26:02.873673 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1425/2825] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jul 14 09:26:02 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 09:26:02.875034 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4887882-2261-4ca8-a277-a0e1bd69b626 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:26:02.887361 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4887882-2261-4ca8-a277-a0e1bd69b626 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:26:02.892404 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a855d960-783e-4e91-9576-d16474be000d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:26:02.893071 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a855d960-783e-4e91-9576-d16474be000d admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:26:02.893353 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a855d960-783e-4e91-9576-d16474be000d admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:26:02.893631 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a855d960-783e-4e91-9576-d16474be000d admin admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:26:02.894391 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a855d960-783e-4e91-9576-d16474be000d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:26:02.896342 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a855d960-783e-4e91-9576-d16474be000d admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:26:02.906677 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1401/2826] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jul 14 09:26:02 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 09:26:02.910731 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bdead4-f8f5-4953-a138-45eedd301d4d admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:26:02.920882 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bdead4-f8f5-4953-a138-45eedd301d4d admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:26:02.926337 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868a11ce-68ae-49b8-9389-b99f0a765a53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:26:02.927221 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868a11ce-68ae-49b8-9389-b99f0a765a53 admin admin] REQUEST_METHOD: `PUT` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:26:02.927501 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868a11ce-68ae-49b8-9389-b99f0a765a53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:26:02.927770 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868a11ce-68ae-49b8-9389-b99f0a765a53 admin admin] PATH_INFO: `/v3/projects/250cb6b3464845868b20cf1f47ab031b/users/9172fa641e5e4226b005d562e72d3cd8/roles/d1691322ce6a4edab7ce722edf0f2988` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:26:02.942993 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868a11ce-68ae-49b8-9389-b99f0a765a53 admin admin] RBAC: Authorizing `identity:create_grant(project_id=250cb6b3464845868b20cf1f47ab031b, user_id=9172fa641e5e4226b005d562e72d3cd8, role_id=d1691322ce6a4edab7ce722edf0f2988)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:26:02.945706 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868a11ce-68ae-49b8-9389-b99f0a765a53 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:26:02.959970 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1426/2827] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jul 14 09:26:02 2022] PUT /identity/v3/projects/250cb6b3464845868b20cf1f47ab031b/users/9172fa641e5e4226b005d562e72d3cd8/roles/d1691322ce6a4edab7ce722edf0f2988 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:26:02.965128 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a277b4-0c04-45ff-b8cf-3d0780684124 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:26:02.965416 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a277b4-0c04-45ff-b8cf-3d0780684124 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:26:02.965692 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a277b4-0c04-45ff-b8cf-3d0780684124 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:26:02.984018 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-19a277b4-0c04-45ff-b8cf-3d0780684124 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:26:02.985229 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-19a277b4-0c04-45ff-b8cf-3d0780684124 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:26:03.046730 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1402/2828] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:26:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:26:03.056444 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868a11ce-68ae-49b8-9389-b99f0a765a53 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:26:03.058264 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-868a11ce-68ae-49b8-9389-b99f0a765a53 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:26:03.071831 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:26:03.117654 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868a11ce-68ae-49b8-9389-b99f0a765a53 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:26:03.121783 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef9e20b-202e-4498-8053-92bcf5f6d6b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '952911c492444ffea4b211a24b564117', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:26:03.122430 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef9e20b-202e-4498-8053-92bcf5f6d6b0 service neutron] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:26:03.122711 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef9e20b-202e-4498-8053-92bcf5f6d6b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:26:03.122984 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef9e20b-202e-4498-8053-92bcf5f6d6b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:26:03.132598 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef9e20b-202e-4498-8053-92bcf5f6d6b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:26:03.134110 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef9e20b-202e-4498-8053-92bcf5f6d6b0 service neutron] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:26:03.143517 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1427/2829] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:26:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:26:03.802009 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8c5659-560b-4696-950d-d2a9e11e9688 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:26:03.802458 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8c5659-560b-4696-950d-d2a9e11e9688 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:26:03.802601 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8c5659-560b-4696-950d-d2a9e11e9688 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:26:03.850163 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-4d8c5659-560b-4696-950d-d2a9e11e9688 None None] MFA Rules not processed for user `9172fa641e5e4226b005d562e72d3cd8`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:26:03.851000 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-4d8c5659-560b-4696-950d-d2a9e11e9688 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:26:03.939304 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1403/2830] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:26:03 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:26:03.949898 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef9e20b-202e-4498-8053-92bcf5f6d6b0 service neutron] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:26:03.951368 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-8ef9e20b-202e-4498-8053-92bcf5f6d6b0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:26:04.008950 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef9e20b-202e-4498-8053-92bcf5f6d6b0 service neutron] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:26:04.012782 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebee9d0-b856-4af3-bb34-96ef8f55f8a8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e76f1952a5b146808844b9f2b7395035', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:26:04.013463 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebee9d0-b856-4af3-bb34-96ef8f55f8a8 service cinder] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:26:04.013727 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebee9d0-b856-4af3-bb34-96ef8f55f8a8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:26:04.014030 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebee9d0-b856-4af3-bb34-96ef8f55f8a8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:26:04.023233 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebee9d0-b856-4af3-bb34-96ef8f55f8a8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:26:04.024672 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebee9d0-b856-4af3-bb34-96ef8f55f8a8 service cinder] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:26:04.033534 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1428/2831] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:26:03 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:26:05.417816 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7c82cb-7404-4c3e-9a39-13a0b1f6d134 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:26:05.418365 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7c82cb-7404-4c3e-9a39-13a0b1f6d134 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:26:05.418365 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7c82cb-7404-4c3e-9a39-13a0b1f6d134 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:26:05.437306 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-7b7c82cb-7404-4c3e-9a39-13a0b1f6d134 None None] MFA Rules not processed for user `9172fa641e5e4226b005d562e72d3cd8`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:26:05.438595 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-7b7c82cb-7404-4c3e-9a39-13a0b1f6d134 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:26:05.446870 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1404/2832] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:26:05 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:26:08.271361 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebee9d0-b856-4af3-bb34-96ef8f55f8a8 service cinder] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:26:08.273091 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-4ebee9d0-b856-4af3-bb34-96ef8f55f8a8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:26:08.292336 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebee9d0-b856-4af3-bb34-96ef8f55f8a8 service cinder] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:26:08.298906 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcc4740-df28-40bb-8ef6-e0bf2196dcc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:26:08.299808 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcc4740-df28-40bb-8ef6-e0bf2196dcc7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:26:08.300126 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcc4740-df28-40bb-8ef6-e0bf2196dcc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:26:08.300411 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcc4740-df28-40bb-8ef6-e0bf2196dcc7 admin admin] PATH_INFO: `/v3/users/5b511b3f04d54edd8e694431c77b0e1b` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:26:08.301794 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcc4740-df28-40bb-8ef6-e0bf2196dcc7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5b511b3f04d54edd8e694431c77b0e1b)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:26:08.304599 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcc4740-df28-40bb-8ef6-e0bf2196dcc7 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:26:08.355791 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-0fcc4740-df28-40bb-8ef6-e0bf2196dcc7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5b511b3f04d54edd8e694431c77b0e1b'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:26:08.356465 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-0fcc4740-df28-40bb-8ef6-e0bf2196dcc7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5b511b3f04d54edd8e694431c77b0e1b'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:26:08.368445 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-0fcc4740-df28-40bb-8ef6-e0bf2196dcc7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5b511b3f04d54edd8e694431c77b0e1b'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:26:08.376645 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-0fcc4740-df28-40bb-8ef6-e0bf2196dcc7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5b511b3f04d54edd8e694431c77b0e1b'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:26:08.377329 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-0fcc4740-df28-40bb-8ef6-e0bf2196dcc7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5b511b3f04d54edd8e694431c77b0e1b'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:26:08.389503 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1429/2833] 10.222.0.26 () {58 vars in 1223 bytes} [Thu Jul 14 09:26:08 2022] DELETE /identity/v3/users/5b511b3f04d54edd8e694431c77b0e1b => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:26:08.394009 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100cb7a5-7e04-4fd6-8046-ac04533e48bb None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:26:08.394475 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100cb7a5-7e04-4fd6-8046-ac04533e48bb None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:26:08.394571 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100cb7a5-7e04-4fd6-8046-ac04533e48bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:26:08.412089 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-100cb7a5-7e04-4fd6-8046-ac04533e48bb None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:26:08.413274 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-100cb7a5-7e04-4fd6-8046-ac04533e48bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:26:08.445683 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:26:08.478476 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1405/2834] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:26:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:26:08.490041 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcc4740-df28-40bb-8ef6-e0bf2196dcc7 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:26:08.491772 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-0fcc4740-df28-40bb-8ef6-e0bf2196dcc7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:26:08.528565 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:26:08.556952 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcc4740-df28-40bb-8ef6-e0bf2196dcc7 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:26:08.560843 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2ac5c7-0250-4d2f-abff-523daa2d1052 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '952911c492444ffea4b211a24b564117', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:26:08.561489 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2ac5c7-0250-4d2f-abff-523daa2d1052 service neutron] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:26:08.561776 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2ac5c7-0250-4d2f-abff-523daa2d1052 service neutron] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:26:08.562044 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2ac5c7-0250-4d2f-abff-523daa2d1052 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:26:08.571389 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2ac5c7-0250-4d2f-abff-523daa2d1052 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:26:08.572826 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2ac5c7-0250-4d2f-abff-523daa2d1052 service neutron] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:26:08.582320 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1430/2835] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:26:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:26:08.856245 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100cb7a5-7e04-4fd6-8046-ac04533e48bb None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:26:08.857723 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-100cb7a5-7e04-4fd6-8046-ac04533e48bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:26:08.870187 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100cb7a5-7e04-4fd6-8046-ac04533e48bb None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:26:08.875053 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a2695a-c83b-4f76-861f-3b44c4d4b9ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:26:08.875973 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a2695a-c83b-4f76-861f-3b44c4d4b9ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:26:08.876252 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a2695a-c83b-4f76-861f-3b44c4d4b9ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:26:08.876531 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a2695a-c83b-4f76-861f-3b44c4d4b9ea admin admin] PATH_INFO: `/v3/projects/c58b6f27d77e4223b86fc424e4b21f9b` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:26:08.877708 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a2695a-c83b-4f76-861f-3b44c4d4b9ea admin admin] RBAC: Authorizing `identity:delete_project(project_id=c58b6f27d77e4223b86fc424e4b21f9b)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:26:08.879675 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a2695a-c83b-4f76-861f-3b44c4d4b9ea admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:26:08.930416 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-d0a2695a-c83b-4f76-861f-3b44c4d4b9ea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c58b6f27d77e4223b86fc424e4b21f9b', 'request_id': 'req-d0a2695a-c83b-4f76-861f-3b44c4d4b9ea'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:26:08.936589 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-d0a2695a-c83b-4f76-861f-3b44c4d4b9ea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c58b6f27d77e4223b86fc424e4b21f9b', 'request_id': 'req-d0a2695a-c83b-4f76-861f-3b44c4d4b9ea'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:26:08.942509 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-d0a2695a-c83b-4f76-861f-3b44c4d4b9ea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c58b6f27d77e4223b86fc424e4b21f9b', 'request_id': 'req-d0a2695a-c83b-4f76-861f-3b44c4d4b9ea'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:26:08.956167 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-d0a2695a-c83b-4f76-861f-3b44c4d4b9ea admin admin] The token cache is being invalidate because project c58b6f27d77e4223b86fc424e4b21f9b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80381) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 09:26:08.956481 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-d0a2695a-c83b-4f76-861f-3b44c4d4b9ea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:26:08.957097 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-d0a2695a-c83b-4f76-861f-3b44c4d4b9ea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:26:08.958704 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1406/2836] 10.222.0.26 () {58 vars in 1232 bytes} [Thu Jul 14 09:26:08 2022] DELETE /identity/v3/projects/c58b6f27d77e4223b86fc424e4b21f9b => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:26:08.963751 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81989fa0-4745-4c83-b5d8-8c0d74f9f162 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:26:08.964039 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81989fa0-4745-4c83-b5d8-8c0d74f9f162 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:26:08.964316 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81989fa0-4745-4c83-b5d8-8c0d74f9f162 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:26:08.983984 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-81989fa0-4745-4c83-b5d8-8c0d74f9f162 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:26:08.985188 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-81989fa0-4745-4c83-b5d8-8c0d74f9f162 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:26:09.051017 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1431/2837] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:26:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:26:09.059020 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbe5b6f-9719-48d7-bcdd-38d8124615f8 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:26:09.059447 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbe5b6f-9719-48d7-bcdd-38d8124615f8 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:26:09.059730 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbe5b6f-9719-48d7-bcdd-38d8124615f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:26:09.078227 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-ddbe5b6f-9719-48d7-bcdd-38d8124615f8 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:26:09.079448 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-ddbe5b6f-9719-48d7-bcdd-38d8124615f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:26:09.088107 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1407/2838] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:26:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:26:09.092314 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81989fa0-4745-4c83-b5d8-8c0d74f9f162 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:26:09.102086 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81989fa0-4745-4c83-b5d8-8c0d74f9f162 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:26:09.107259 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210fff30-5cf7-49e0-b9a2-5e1ce5b3f6bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:26:09.107912 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210fff30-5cf7-49e0-b9a2-5e1ce5b3f6bf admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:26:09.108197 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210fff30-5cf7-49e0-b9a2-5e1ce5b3f6bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:26:09.108468 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210fff30-5cf7-49e0-b9a2-5e1ce5b3f6bf admin admin] PATH_INFO: `/v3/domains` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:26:09.109189 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210fff30-5cf7-49e0-b9a2-5e1ce5b3f6bf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:26:09.110368 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210fff30-5cf7-49e0-b9a2-5e1ce5b3f6bf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:26:09.111118 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210fff30-5cf7-49e0-b9a2-5e1ce5b3f6bf admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:26:09.122765 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1432/2839] 10.222.0.26 () {58 vars in 1152 bytes} [Thu Jul 14 09:26:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:26:09.127074 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbe5b6f-9719-48d7-bcdd-38d8124615f8 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:26:09.136337 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbe5b6f-9719-48d7-bcdd-38d8124615f8 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:26:09.141200 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c8f5ae-0f94-4aea-8c93-d5991f781918 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:26:09.141842 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c8f5ae-0f94-4aea-8c93-d5991f781918 admin admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:26:09.142124 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c8f5ae-0f94-4aea-8c93-d5991f781918 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:26:09.142391 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c8f5ae-0f94-4aea-8c93-d5991f781918 admin admin] PATH_INFO: `/v3/projects` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:26:09.143080 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c8f5ae-0f94-4aea-8c93-d5991f781918 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:26:09.145021 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c8f5ae-0f94-4aea-8c93-d5991f781918 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:26:09.174017 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1408/2840] 10.222.0.26 () {60 vars in 1152 bytes} [Thu Jul 14 09:26:09 2022] POST /identity/v3/projects => generated 363 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 09:26:09.177933 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210fff30-5cf7-49e0-b9a2-5e1ce5b3f6bf admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:26:09.188241 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210fff30-5cf7-49e0-b9a2-5e1ce5b3f6bf admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:26:09.247353 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845f5169-0f91-4e99-9359-4573a0f1423c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:26:09.248031 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845f5169-0f91-4e99-9359-4573a0f1423c admin admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:26:09.248313 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845f5169-0f91-4e99-9359-4573a0f1423c admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:26:09.248584 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845f5169-0f91-4e99-9359-4573a0f1423c admin admin] PATH_INFO: `/v3/users` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:26:09.249282 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845f5169-0f91-4e99-9359-4573a0f1423c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:26:09.251229 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845f5169-0f91-4e99-9359-4573a0f1423c admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:26:09.272940 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1433/2841] 10.222.0.26 () {60 vars in 1143 bytes} [Thu Jul 14 09:26:09 2022] POST /identity/v3/users => generated 282 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 09:26:09.277275 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c8f5ae-0f94-4aea-8c93-d5991f781918 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:26:09.287514 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c8f5ae-0f94-4aea-8c93-d5991f781918 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:26:09.292187 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2f8571-3d5c-47a9-a55c-0c43f2ed5caa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:26:09.292845 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2f8571-3d5c-47a9-a55c-0c43f2ed5caa admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:26:09.293129 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2f8571-3d5c-47a9-a55c-0c43f2ed5caa admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:26:09.293405 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2f8571-3d5c-47a9-a55c-0c43f2ed5caa admin admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:26:09.294137 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2f8571-3d5c-47a9-a55c-0c43f2ed5caa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:26:09.295798 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2f8571-3d5c-47a9-a55c-0c43f2ed5caa admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:26:09.306201 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1409/2842] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jul 14 09:26:09 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 09:26:09.309874 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845f5169-0f91-4e99-9359-4573a0f1423c admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:26:09.319805 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845f5169-0f91-4e99-9359-4573a0f1423c admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:26:09.324477 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235fa399-baa7-4a9a-a8c8-6f14910acd2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:26:09.325134 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235fa399-baa7-4a9a-a8c8-6f14910acd2a admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:26:09.325414 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235fa399-baa7-4a9a-a8c8-6f14910acd2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:26:09.325708 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235fa399-baa7-4a9a-a8c8-6f14910acd2a admin admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:26:09.326464 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235fa399-baa7-4a9a-a8c8-6f14910acd2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:26:09.328122 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235fa399-baa7-4a9a-a8c8-6f14910acd2a admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:26:09.338715 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1434/2843] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jul 14 09:26:09 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 09:26:09.342919 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2f8571-3d5c-47a9-a55c-0c43f2ed5caa admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:26:09.353070 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2f8571-3d5c-47a9-a55c-0c43f2ed5caa admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:26:09.357608 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52963fdd-e01d-4369-86f1-a8fdf809bcd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:26:09.358532 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52963fdd-e01d-4369-86f1-a8fdf809bcd7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:26:09.358812 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52963fdd-e01d-4369-86f1-a8fdf809bcd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:26:09.359081 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52963fdd-e01d-4369-86f1-a8fdf809bcd7 admin admin] PATH_INFO: `/v3/projects/8a75e8c5e3914518861e4353b03492fb/users/e9b3bbbc2a584649bf0078c977e7e919/roles/d1691322ce6a4edab7ce722edf0f2988` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:26:09.374253 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52963fdd-e01d-4369-86f1-a8fdf809bcd7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a75e8c5e3914518861e4353b03492fb, user_id=e9b3bbbc2a584649bf0078c977e7e919, role_id=d1691322ce6a4edab7ce722edf0f2988)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:26:09.376965 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52963fdd-e01d-4369-86f1-a8fdf809bcd7 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:26:09.392450 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1410/2844] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jul 14 09:26:09 2022] PUT /identity/v3/projects/8a75e8c5e3914518861e4353b03492fb/users/e9b3bbbc2a584649bf0078c977e7e919/roles/d1691322ce6a4edab7ce722edf0f2988 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 09:26:09.403775 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d87c32a-8e3b-4b1a-b24c-4877edee1726 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:26:09.404061 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d87c32a-8e3b-4b1a-b24c-4877edee1726 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:26:09.404331 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d87c32a-8e3b-4b1a-b24c-4877edee1726 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:26:09.430776 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-4d87c32a-8e3b-4b1a-b24c-4877edee1726 None None] MFA Rules not processed for user `e9b3bbbc2a584649bf0078c977e7e919`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:26:09.432062 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-4d87c32a-8e3b-4b1a-b24c-4877edee1726 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:26:09.458895 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:26:09.498869 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1435/2845] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:26:09 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:26:09.510792 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52963fdd-e01d-4369-86f1-a8fdf809bcd7 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:26:09.512770 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-52963fdd-e01d-4369-86f1-a8fdf809bcd7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:26:09.580755 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:26:09.598549 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52963fdd-e01d-4369-86f1-a8fdf809bcd7 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:26:09.603394 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6551852a-8860-4ca7-bfea-578e4086ff7f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e76f1952a5b146808844b9f2b7395035', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:26:09.604043 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6551852a-8860-4ca7-bfea-578e4086ff7f service cinder] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:26:09.604323 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6551852a-8860-4ca7-bfea-578e4086ff7f service cinder] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:26:09.604592 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6551852a-8860-4ca7-bfea-578e4086ff7f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:26:09.614135 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6551852a-8860-4ca7-bfea-578e4086ff7f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:26:09.615598 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6551852a-8860-4ca7-bfea-578e4086ff7f service cinder] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:26:09.627609 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1411/2846] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:26:09 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:26:19.495293 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=15 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:26:19.621678 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=9 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:26:19.786146 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d87c32a-8e3b-4b1a-b24c-4877edee1726 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:26:19.788420 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-4d87c32a-8e3b-4b1a-b24c-4877edee1726 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:26:19.856550 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d87c32a-8e3b-4b1a-b24c-4877edee1726 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:26:19.861579 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3dfbd5-a782-473a-aa46-8387b60998a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:26:19.862453 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3dfbd5-a782-473a-aa46-8387b60998a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:26:19.862732 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3dfbd5-a782-473a-aa46-8387b60998a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:26:19.863002 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3dfbd5-a782-473a-aa46-8387b60998a1 admin admin] PATH_INFO: `/v3/users/ce70deb538b046d9b45cd804d13e373d` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:26:19.864217 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3dfbd5-a782-473a-aa46-8387b60998a1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ce70deb538b046d9b45cd804d13e373d)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:26:19.866138 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3dfbd5-a782-473a-aa46-8387b60998a1 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:26:19.915406 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-0d3dfbd5-a782-473a-aa46-8387b60998a1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ce70deb538b046d9b45cd804d13e373d'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:26:19.916042 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-0d3dfbd5-a782-473a-aa46-8387b60998a1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ce70deb538b046d9b45cd804d13e373d'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:26:19.928183 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-0d3dfbd5-a782-473a-aa46-8387b60998a1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ce70deb538b046d9b45cd804d13e373d'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:26:19.936281 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-0d3dfbd5-a782-473a-aa46-8387b60998a1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ce70deb538b046d9b45cd804d13e373d'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:26:19.936952 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-0d3dfbd5-a782-473a-aa46-8387b60998a1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ce70deb538b046d9b45cd804d13e373d'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:26:19.951370 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1436/2847] 10.222.0.26 () {58 vars in 1223 bytes} [Thu Jul 14 09:26:19 2022] DELETE /identity/v3/users/ce70deb538b046d9b45cd804d13e373d => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:26:19.956956 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fb1e5d-b75b-47d1-bc89-7bc2d2af35c3 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:26:19.957244 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fb1e5d-b75b-47d1-bc89-7bc2d2af35c3 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:26:19.957521 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fb1e5d-b75b-47d1-bc89-7bc2d2af35c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:26:19.976845 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-52fb1e5d-b75b-47d1-bc89-7bc2d2af35c3 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:26:19.978055 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-52fb1e5d-b75b-47d1-bc89-7bc2d2af35c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:26:20.040526 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1412/2848] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:26:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:26:20.050312 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3dfbd5-a782-473a-aa46-8387b60998a1 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:26:20.051837 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-0d3dfbd5-a782-473a-aa46-8387b60998a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:26:20.113514 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3dfbd5-a782-473a-aa46-8387b60998a1 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:26:20.117366 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34e4d1e-ca3f-4ac6-9b6f-9c6fdcb79b8e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '952911c492444ffea4b211a24b564117', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:26:20.118022 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34e4d1e-ca3f-4ac6-9b6f-9c6fdcb79b8e service neutron] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:26:20.118304 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34e4d1e-ca3f-4ac6-9b6f-9c6fdcb79b8e service neutron] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:26:20.118579 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34e4d1e-ca3f-4ac6-9b6f-9c6fdcb79b8e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:26:20.128288 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34e4d1e-ca3f-4ac6-9b6f-9c6fdcb79b8e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:26:20.129747 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34e4d1e-ca3f-4ac6-9b6f-9c6fdcb79b8e service neutron] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:26:20.139615 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1437/2849] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:26:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:26:20.442454 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fb1e5d-b75b-47d1-bc89-7bc2d2af35c3 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:26:20.443907 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-52fb1e5d-b75b-47d1-bc89-7bc2d2af35c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:26:20.464886 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fb1e5d-b75b-47d1-bc89-7bc2d2af35c3 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:26:20.472167 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386dfe3c-d08f-4fbe-a239-69687b270bd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:26:20.472941 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386dfe3c-d08f-4fbe-a239-69687b270bd6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:26:20.473321 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386dfe3c-d08f-4fbe-a239-69687b270bd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:26:20.473495 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386dfe3c-d08f-4fbe-a239-69687b270bd6 admin admin] PATH_INFO: `/v3/projects/e633ca032ee7404699455c896ad8a31b` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:26:20.474879 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386dfe3c-d08f-4fbe-a239-69687b270bd6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e633ca032ee7404699455c896ad8a31b)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:26:20.477190 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386dfe3c-d08f-4fbe-a239-69687b270bd6 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:26:20.529575 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-386dfe3c-d08f-4fbe-a239-69687b270bd6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e633ca032ee7404699455c896ad8a31b', 'request_id': 'req-386dfe3c-d08f-4fbe-a239-69687b270bd6'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:26:20.536599 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-386dfe3c-d08f-4fbe-a239-69687b270bd6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e633ca032ee7404699455c896ad8a31b', 'request_id': 'req-386dfe3c-d08f-4fbe-a239-69687b270bd6'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:26:20.542281 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-386dfe3c-d08f-4fbe-a239-69687b270bd6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e633ca032ee7404699455c896ad8a31b', 'request_id': 'req-386dfe3c-d08f-4fbe-a239-69687b270bd6'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:26:20.558677 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-386dfe3c-d08f-4fbe-a239-69687b270bd6 admin admin] The token cache is being invalidate because project e633ca032ee7404699455c896ad8a31b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80381) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 09:26:20.558966 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-386dfe3c-d08f-4fbe-a239-69687b270bd6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:26:20.559615 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-386dfe3c-d08f-4fbe-a239-69687b270bd6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:26:20.561180 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1413/2850] 10.222.0.26 () {58 vars in 1232 bytes} [Thu Jul 14 09:26:20 2022] DELETE /identity/v3/projects/e633ca032ee7404699455c896ad8a31b => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:26:27.597846 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34e4d1e-ca3f-4ac6-9b6f-9c6fdcb79b8e service neutron] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:26:27.599417 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-e34e4d1e-ca3f-4ac6-9b6f-9c6fdcb79b8e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:26:27.636963 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:26:27.708761 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34e4d1e-ca3f-4ac6-9b6f-9c6fdcb79b8e service neutron] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:26:27.789089 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df88d91-63fb-4e79-aa11-30595e3d5d52 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '355684f52e074ad9bfa7fd90bcae6caf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:26:27.789772 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df88d91-63fb-4e79-aa11-30595e3d5d52 service placement] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:26:27.790164 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df88d91-63fb-4e79-aa11-30595e3d5d52 service placement] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:26:27.790443 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df88d91-63fb-4e79-aa11-30595e3d5d52 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:26:27.792481 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-1df88d91-63fb-4e79-aa11-30595e3d5d52 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:26:27.806159 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df88d91-63fb-4e79-aa11-30595e3d5d52 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:26:27.807991 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df88d91-63fb-4e79-aa11-30595e3d5d52 service placement] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:26:27.819734 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1438/2851] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:26:27 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:26:27.990984 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386dfe3c-d08f-4fbe-a239-69687b270bd6 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:26:27.992475 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-386dfe3c-d08f-4fbe-a239-69687b270bd6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:26:28.061069 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386dfe3c-d08f-4fbe-a239-69687b270bd6 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:26:28.065854 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b9fc24-2c86-48d9-bb8e-c1d235ffd440 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:26:28.066849 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b9fc24-2c86-48d9-bb8e-c1d235ffd440 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:26:28.067130 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b9fc24-2c86-48d9-bb8e-c1d235ffd440 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:26:28.067445 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b9fc24-2c86-48d9-bb8e-c1d235ffd440 admin admin] PATH_INFO: `/v3/users/42662fb9bae5403a8a6f698358f5c20c` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:26:28.068664 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b9fc24-2c86-48d9-bb8e-c1d235ffd440 admin admin] RBAC: Authorizing `identity:delete_user(user_id=42662fb9bae5403a8a6f698358f5c20c)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:26:28.070819 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b9fc24-2c86-48d9-bb8e-c1d235ffd440 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:26:28.102032 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:26:28.128744 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-e8b9fc24-2c86-48d9-bb8e-c1d235ffd440 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '42662fb9bae5403a8a6f698358f5c20c'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:26:28.138827 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-e8b9fc24-2c86-48d9-bb8e-c1d235ffd440 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '42662fb9bae5403a8a6f698358f5c20c'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:26:28.139596 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-e8b9fc24-2c86-48d9-bb8e-c1d235ffd440 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '42662fb9bae5403a8a6f698358f5c20c'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:26:28.140309 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-e8b9fc24-2c86-48d9-bb8e-c1d235ffd440 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '42662fb9bae5403a8a6f698358f5c20c'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:26:28.150343 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-e8b9fc24-2c86-48d9-bb8e-c1d235ffd440 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '42662fb9bae5403a8a6f698358f5c20c'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:26:28.170354 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1414/2852] 10.222.0.26 () {58 vars in 1223 bytes} [Thu Jul 14 09:26:27 2022] DELETE /identity/v3/users/42662fb9bae5403a8a6f698358f5c20c => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:26:28.363929 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df88d91-63fb-4e79-aa11-30595e3d5d52 service placement] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:26:28.378793 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df88d91-63fb-4e79-aa11-30595e3d5d52 service placement] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:26:28.459926 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f85224-6026-450e-8511-3e249ced256b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:26:28.460843 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f85224-6026-450e-8511-3e249ced256b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:26:28.461123 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f85224-6026-450e-8511-3e249ced256b admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:26:28.461395 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f85224-6026-450e-8511-3e249ced256b admin admin] PATH_INFO: `/v3/projects/335a0619bd62458783523da604a22d40` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:26:28.462619 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f85224-6026-450e-8511-3e249ced256b admin admin] RBAC: Authorizing `identity:delete_project(project_id=335a0619bd62458783523da604a22d40)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:26:28.464573 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f85224-6026-450e-8511-3e249ced256b admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:26:28.504595 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:26:28.514950 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-57f85224-6026-450e-8511-3e249ced256b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '335a0619bd62458783523da604a22d40', 'request_id': 'req-57f85224-6026-450e-8511-3e249ced256b'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:26:28.522138 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-57f85224-6026-450e-8511-3e249ced256b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '335a0619bd62458783523da604a22d40', 'request_id': 'req-57f85224-6026-450e-8511-3e249ced256b'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:26:28.529137 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-57f85224-6026-450e-8511-3e249ced256b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '335a0619bd62458783523da604a22d40', 'request_id': 'req-57f85224-6026-450e-8511-3e249ced256b'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:26:28.544421 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-57f85224-6026-450e-8511-3e249ced256b admin admin] The token cache is being invalidate because project 335a0619bd62458783523da604a22d40 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80382) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 09:26:28.544980 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-57f85224-6026-450e-8511-3e249ced256b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:26:28.545774 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-57f85224-6026-450e-8511-3e249ced256b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:26:28.548204 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1439/2853] 10.222.0.26 () {58 vars in 1232 bytes} [Thu Jul 14 09:26:28 2022] DELETE /identity/v3/projects/335a0619bd62458783523da604a22d40 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:26:28.554173 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cb39bb-c091-4536-863b-8ff5ccd83443 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:26:28.554531 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cb39bb-c091-4536-863b-8ff5ccd83443 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:26:28.554738 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cb39bb-c091-4536-863b-8ff5ccd83443 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:26:28.575529 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-77cb39bb-c091-4536-863b-8ff5ccd83443 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:26:28.576802 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-77cb39bb-c091-4536-863b-8ff5ccd83443 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:26:28.647754 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1415/2854] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:26:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:26:28.659087 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4028d9e9-38e2-4bb6-befe-ec3ea455ca94 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:26:28.660105 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4028d9e9-38e2-4bb6-befe-ec3ea455ca94 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:26:28.660105 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4028d9e9-38e2-4bb6-befe-ec3ea455ca94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:26:28.680283 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-4028d9e9-38e2-4bb6-befe-ec3ea455ca94 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:26:28.681492 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-4028d9e9-38e2-4bb6-befe-ec3ea455ca94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:26:28.691224 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1440/2855] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:26:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:26:28.696358 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cb39bb-c091-4536-863b-8ff5ccd83443 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:26:28.712954 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cb39bb-c091-4536-863b-8ff5ccd83443 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:26:28.721425 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6c1343-3662-4d47-9854-58c091d91f57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:26:28.722589 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6c1343-3662-4d47-9854-58c091d91f57 admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:26:28.722922 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6c1343-3662-4d47-9854-58c091d91f57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:26:28.723233 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6c1343-3662-4d47-9854-58c091d91f57 admin admin] PATH_INFO: `/v3/domains` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:26:28.724060 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6c1343-3662-4d47-9854-58c091d91f57 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:26:28.724382 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6c1343-3662-4d47-9854-58c091d91f57 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:26:28.726152 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6c1343-3662-4d47-9854-58c091d91f57 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:26:28.740450 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1416/2856] 10.222.0.26 () {58 vars in 1152 bytes} [Thu Jul 14 09:26:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:26:28.745575 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4028d9e9-38e2-4bb6-befe-ec3ea455ca94 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:26:28.755519 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4028d9e9-38e2-4bb6-befe-ec3ea455ca94 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:26:28.761721 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f74b0f-29b9-4e26-8d57-5b82a92bb073 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:26:28.762743 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f74b0f-29b9-4e26-8d57-5b82a92bb073 admin admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:26:28.763044 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f74b0f-29b9-4e26-8d57-5b82a92bb073 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:26:28.763381 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f74b0f-29b9-4e26-8d57-5b82a92bb073 admin admin] PATH_INFO: `/v3/projects` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:26:28.764106 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f74b0f-29b9-4e26-8d57-5b82a92bb073 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:26:28.766118 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f74b0f-29b9-4e26-8d57-5b82a92bb073 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:26:28.796840 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1441/2857] 10.222.0.26 () {60 vars in 1152 bytes} [Thu Jul 14 09:26:28 2022] POST /identity/v3/projects => generated 373 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 09:26:28.801405 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6c1343-3662-4d47-9854-58c091d91f57 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:26:28.812434 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6c1343-3662-4d47-9854-58c091d91f57 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:26:28.873240 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cce8e3-6e54-4f59-8880-b83911450ce0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:26:28.873919 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cce8e3-6e54-4f59-8880-b83911450ce0 admin admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:26:28.874530 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cce8e3-6e54-4f59-8880-b83911450ce0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:26:28.874530 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cce8e3-6e54-4f59-8880-b83911450ce0 admin admin] PATH_INFO: `/v3/users` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:26:28.875234 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cce8e3-6e54-4f59-8880-b83911450ce0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:26:28.877252 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cce8e3-6e54-4f59-8880-b83911450ce0 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:26:28.900821 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1417/2858] 10.222.0.26 () {60 vars in 1143 bytes} [Thu Jul 14 09:26:28 2022] POST /identity/v3/users => generated 287 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 09:26:28.908490 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f74b0f-29b9-4e26-8d57-5b82a92bb073 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:26:28.917244 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58270104-8101-4047-89ee-4c89407b6d88 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:26:28.917534 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58270104-8101-4047-89ee-4c89407b6d88 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:26:28.917814 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58270104-8101-4047-89ee-4c89407b6d88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:26:28.918436 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f74b0f-29b9-4e26-8d57-5b82a92bb073 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:26:28.925729 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424eaf75-ff92-43f0-ad9f-8be523c645f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:26:28.926419 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424eaf75-ff92-43f0-ad9f-8be523c645f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:26:28.926696 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424eaf75-ff92-43f0-ad9f-8be523c645f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:26:28.926963 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424eaf75-ff92-43f0-ad9f-8be523c645f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:26:28.927758 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424eaf75-ff92-43f0-ad9f-8be523c645f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:26:28.929398 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424eaf75-ff92-43f0-ad9f-8be523c645f8 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:26:28.937828 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-58270104-8101-4047-89ee-4c89407b6d88 None None] MFA Rules not processed for user `9172fa641e5e4226b005d562e72d3cd8`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:26:28.939076 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-58270104-8101-4047-89ee-4c89407b6d88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:26:28.942858 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1442/2859] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jul 14 09:26:28 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 09:26:28.947136 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424eaf75-ff92-43f0-ad9f-8be523c645f8 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:26:28.956132 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:26:28.957655 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424eaf75-ff92-43f0-ad9f-8be523c645f8 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:26:28.962848 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b555ea5e-3350-4177-8f85-5aabc9bdef07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:26:28.963707 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b555ea5e-3350-4177-8f85-5aabc9bdef07 admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:26:28.963998 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b555ea5e-3350-4177-8f85-5aabc9bdef07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:26:28.964276 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b555ea5e-3350-4177-8f85-5aabc9bdef07 admin admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:26:28.965094 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b555ea5e-3350-4177-8f85-5aabc9bdef07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:26:28.966854 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b555ea5e-3350-4177-8f85-5aabc9bdef07 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:26:28.978170 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1443/2860] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jul 14 09:26:28 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 09:26:28.986995 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b555ea5e-3350-4177-8f85-5aabc9bdef07 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:26:28.997899 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b555ea5e-3350-4177-8f85-5aabc9bdef07 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:26:29.003865 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f7a8b7-ac9f-4180-b4c2-1af2fa7ab0c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:26:29.004808 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f7a8b7-ac9f-4180-b4c2-1af2fa7ab0c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:26:29.005107 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f7a8b7-ac9f-4180-b4c2-1af2fa7ab0c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:26:29.005382 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f7a8b7-ac9f-4180-b4c2-1af2fa7ab0c9 admin admin] PATH_INFO: `/v3/projects/8e3d77e253bd4167b4be5c923e026228/users/56bd87cf833847cc9c8b5e78348de6fc/roles/d1691322ce6a4edab7ce722edf0f2988` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:26:29.009336 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1418/2861] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:26:28 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:26:29.022006 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f7a8b7-ac9f-4180-b4c2-1af2fa7ab0c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e3d77e253bd4167b4be5c923e026228, user_id=56bd87cf833847cc9c8b5e78348de6fc, role_id=d1691322ce6a4edab7ce722edf0f2988)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:26:29.024690 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f7a8b7-ac9f-4180-b4c2-1af2fa7ab0c9 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:26:29.039679 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1444/2862] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jul 14 09:26:28 2022] PUT /identity/v3/projects/8e3d77e253bd4167b4be5c923e026228/users/56bd87cf833847cc9c8b5e78348de6fc/roles/d1691322ce6a4edab7ce722edf0f2988 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 09:26:29.044635 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a00a3d-2ce1-48aa-bb86-a1e0fdf9f07a None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:26:29.045000 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a00a3d-2ce1-48aa-bb86-a1e0fdf9f07a None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:26:29.045285 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a00a3d-2ce1-48aa-bb86-a1e0fdf9f07a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:26:29.065233 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-39a00a3d-2ce1-48aa-bb86-a1e0fdf9f07a None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:26:29.066490 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-39a00a3d-2ce1-48aa-bb86-a1e0fdf9f07a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:26:29.137207 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1419/2863] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:26:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:26:29.147791 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f7a8b7-ac9f-4180-b4c2-1af2fa7ab0c9 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:26:29.149593 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-50f7a8b7-ac9f-4180-b4c2-1af2fa7ab0c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:26:29.226938 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f7a8b7-ac9f-4180-b4c2-1af2fa7ab0c9 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:26:29.230936 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5db30c9-3cd4-4516-ae53-283b5de2d77f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '952911c492444ffea4b211a24b564117', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:26:29.231636 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5db30c9-3cd4-4516-ae53-283b5de2d77f service neutron] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:26:29.231922 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5db30c9-3cd4-4516-ae53-283b5de2d77f service neutron] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:26:29.232205 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5db30c9-3cd4-4516-ae53-283b5de2d77f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:26:29.241994 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5db30c9-3cd4-4516-ae53-283b5de2d77f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:26:29.243503 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5db30c9-3cd4-4516-ae53-283b5de2d77f service neutron] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:26:29.253699 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1445/2864] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:26:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:26:33.991069 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f029fa67-ac47-4149-9ed4-deec0471652d None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:26:33.991991 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f029fa67-ac47-4149-9ed4-deec0471652d None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:26:33.992420 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f029fa67-ac47-4149-9ed4-deec0471652d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:26:34.027398 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-f029fa67-ac47-4149-9ed4-deec0471652d None None] MFA Rules not processed for user `56bd87cf833847cc9c8b5e78348de6fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:26:34.028847 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-f029fa67-ac47-4149-9ed4-deec0471652d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:26:34.117920 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1420/2865] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:26:33 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:26:34.119534 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5db30c9-3cd4-4516-ae53-283b5de2d77f service neutron] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:26:34.131155 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5db30c9-3cd4-4516-ae53-283b5de2d77f service neutron] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:26:34.136928 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e2787b-bbdc-462b-b0e6-4822677d6ed4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:26:34.138387 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e2787b-bbdc-462b-b0e6-4822677d6ed4 admin admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:26:34.138788 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e2787b-bbdc-462b-b0e6-4822677d6ed4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:26:34.139149 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e2787b-bbdc-462b-b0e6-4822677d6ed4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:26:34.139969 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e2787b-bbdc-462b-b0e6-4822677d6ed4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:26:34.142266 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e2787b-bbdc-462b-b0e6-4822677d6ed4 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:26:34.179425 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1446/2866] 10.222.0.26 () {60 vars in 1152 bytes} [Thu Jul 14 09:26:34 2022] POST /identity/v3/projects => generated 373 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 09:26:34.181396 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f029fa67-ac47-4149-9ed4-deec0471652d None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:26:34.195420 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f029fa67-ac47-4149-9ed4-deec0471652d None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:26:34.255247 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:26:34.265745 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5e0dfa-e871-43de-8b16-4eff43b3de57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:26:34.266429 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5e0dfa-e871-43de-8b16-4eff43b3de57 admin admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:26:34.266714 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5e0dfa-e871-43de-8b16-4eff43b3de57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:26:34.266993 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5e0dfa-e871-43de-8b16-4eff43b3de57 admin admin] PATH_INFO: `/v3/users` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:26:34.267742 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5e0dfa-e871-43de-8b16-4eff43b3de57 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:26:34.269801 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5e0dfa-e871-43de-8b16-4eff43b3de57 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:26:34.297294 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1421/2867] 10.222.0.26 () {60 vars in 1143 bytes} [Thu Jul 14 09:26:34 2022] POST /identity/v3/users => generated 293 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 09:26:34.298555 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e2787b-bbdc-462b-b0e6-4822677d6ed4 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:26:34.311313 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e2787b-bbdc-462b-b0e6-4822677d6ed4 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:26:34.316507 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e0cdbf-e7f2-4025-8bbb-e1257c4beb65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:26:34.317275 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e0cdbf-e7f2-4025-8bbb-e1257c4beb65 admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:26:34.317626 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e0cdbf-e7f2-4025-8bbb-e1257c4beb65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:26:34.317970 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e0cdbf-e7f2-4025-8bbb-e1257c4beb65 admin admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:26:34.318815 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e0cdbf-e7f2-4025-8bbb-e1257c4beb65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:26:34.321105 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e0cdbf-e7f2-4025-8bbb-e1257c4beb65 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:26:34.334503 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1447/2868] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jul 14 09:26:34 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 09:26:34.339657 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5e0dfa-e871-43de-8b16-4eff43b3de57 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:26:34.351684 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5e0dfa-e871-43de-8b16-4eff43b3de57 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:26:34.358451 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4bc38c-f390-44a0-a72f-69525b96fe8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:26:34.359671 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4bc38c-f390-44a0-a72f-69525b96fe8a admin admin] REQUEST_METHOD: `PUT` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:26:34.360016 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4bc38c-f390-44a0-a72f-69525b96fe8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:26:34.360470 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4bc38c-f390-44a0-a72f-69525b96fe8a admin admin] PATH_INFO: `/v3/projects/00aed77165b04cdd9becc90c19248929/users/02c93dbd9c3c4e51ba19072293cab44b/roles/7b549d58d1724395b343609d895c80d0` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:26:34.378312 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4bc38c-f390-44a0-a72f-69525b96fe8a admin admin] RBAC: Authorizing `identity:create_grant(project_id=00aed77165b04cdd9becc90c19248929, user_id=02c93dbd9c3c4e51ba19072293cab44b, role_id=7b549d58d1724395b343609d895c80d0)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:26:34.384685 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4bc38c-f390-44a0-a72f-69525b96fe8a admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:26:34.400918 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1422/2869] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jul 14 09:26:34 2022] PUT /identity/v3/projects/00aed77165b04cdd9becc90c19248929/users/02c93dbd9c3c4e51ba19072293cab44b/roles/7b549d58d1724395b343609d895c80d0 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:26:34.404547 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e0cdbf-e7f2-4025-8bbb-e1257c4beb65 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:26:34.421865 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e0cdbf-e7f2-4025-8bbb-e1257c4beb65 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:26:34.440568 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:26:34.489011 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6b9948-8a60-4a8d-81ac-0775e4e8fdb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:26:34.489744 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6b9948-8a60-4a8d-81ac-0775e4e8fdb1 admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:26:34.490020 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6b9948-8a60-4a8d-81ac-0775e4e8fdb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:26:34.490665 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6b9948-8a60-4a8d-81ac-0775e4e8fdb1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:26:34.491067 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6b9948-8a60-4a8d-81ac-0775e4e8fdb1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:26:34.492706 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6b9948-8a60-4a8d-81ac-0775e4e8fdb1 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:26:34.506718 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1448/2870] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jul 14 09:26:34 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 09:26:34.510598 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4bc38c-f390-44a0-a72f-69525b96fe8a admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:26:34.523311 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4bc38c-f390-44a0-a72f-69525b96fe8a admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:26:34.534572 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa274c4-e4b3-4f83-83ae-ef63a9653e7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:26:34.535623 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa274c4-e4b3-4f83-83ae-ef63a9653e7e admin admin] REQUEST_METHOD: `PUT` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:26:34.536496 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa274c4-e4b3-4f83-83ae-ef63a9653e7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:26:34.536496 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa274c4-e4b3-4f83-83ae-ef63a9653e7e admin admin] PATH_INFO: `/v3/projects/00aed77165b04cdd9becc90c19248929/users/02c93dbd9c3c4e51ba19072293cab44b/roles/7b549d58d1724395b343609d895c80d0` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:26:34.538334 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa274c4-e4b3-4f83-83ae-ef63a9653e7e admin admin] RBAC: Authorizing `identity:create_grant(project_id=00aed77165b04cdd9becc90c19248929, user_id=02c93dbd9c3c4e51ba19072293cab44b, role_id=7b549d58d1724395b343609d895c80d0)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:26:34.541637 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa274c4-e4b3-4f83-83ae-ef63a9653e7e admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:26:34.578937 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1423/2871] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jul 14 09:26:34 2022] PUT /identity/v3/projects/00aed77165b04cdd9becc90c19248929/users/02c93dbd9c3c4e51ba19072293cab44b/roles/7b549d58d1724395b343609d895c80d0 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:26:40.097714 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d920d4-1b35-45a2-aeef-07a01ffac9ec None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:26:40.098257 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d920d4-1b35-45a2-aeef-07a01ffac9ec None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:26:40.104162 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d920d4-1b35-45a2-aeef-07a01ffac9ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:26:40.140240 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-d5d920d4-1b35-45a2-aeef-07a01ffac9ec None None] MFA Rules not processed for user `02c93dbd9c3c4e51ba19072293cab44b`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:26:40.142191 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-d5d920d4-1b35-45a2-aeef-07a01ffac9ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:26:40.232564 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1449/2872] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:26:40 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:26:40.249307 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa274c4-e4b3-4f83-83ae-ef63a9653e7e admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:26:40.252144 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-caa274c4-e4b3-4f83-83ae-ef63a9653e7e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:26:40.365800 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa274c4-e4b3-4f83-83ae-ef63a9653e7e admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:26:40.371681 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253efd6c-df61-476f-ba77-f3687ad283de service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '538fff3c5a2b487eb44ae7c1a1198102', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:26:40.372332 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253efd6c-df61-476f-ba77-f3687ad283de service nova] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:26:40.372613 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253efd6c-df61-476f-ba77-f3687ad283de service nova] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:26:40.372888 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253efd6c-df61-476f-ba77-f3687ad283de service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:26:40.429128 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253efd6c-df61-476f-ba77-f3687ad283de service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:26:40.430729 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253efd6c-df61-476f-ba77-f3687ad283de service nova] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:26:40.440301 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1424/2873] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:26:40 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:26:41.567080 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e5e162-7c55-4448-884d-26b8de5bc846 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:26:41.567080 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e5e162-7c55-4448-884d-26b8de5bc846 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:26:41.567080 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e5e162-7c55-4448-884d-26b8de5bc846 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:26:41.574102 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-05e5e162-7c55-4448-884d-26b8de5bc846 None None] MFA Rules not processed for user `56bd87cf833847cc9c8b5e78348de6fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:26:41.575542 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-05e5e162-7c55-4448-884d-26b8de5bc846 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:26:41.582998 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1450/2874] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:26:41 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:26:50.436302 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=87,keystone:INSERT=5 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:26:50.819873 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f849b2a0-3131-4b37-acd7-d2ecd2d962f3 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:26:50.820196 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f849b2a0-3131-4b37-acd7-d2ecd2d962f3 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:26:50.820639 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f849b2a0-3131-4b37-acd7-d2ecd2d962f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:26:50.856444 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-f849b2a0-3131-4b37-acd7-d2ecd2d962f3 None None] MFA Rules not processed for user `7d6267ede8d64e2abe1f4058a9797fb7`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:26:50.857656 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-f849b2a0-3131-4b37-acd7-d2ecd2d962f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:26:50.921410 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1425/2875] 10.222.0.26 () {58 vars in 1009 bytes} [Thu Jul 14 09:26:50 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:26:50.972510 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e5e162-7c55-4448-884d-26b8de5bc846 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:26:50.974114 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-05e5e162-7c55-4448-884d-26b8de5bc846 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:26:51.050973 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e5e162-7c55-4448-884d-26b8de5bc846 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:26:51.056475 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3287d0cf-e281-41fc-9901-0acbec0722f8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5a43c80514b4a13af03c7f8ebcf7a5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:26:51.057141 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3287d0cf-e281-41fc-9901-0acbec0722f8 service swift] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:26:51.057421 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3287d0cf-e281-41fc-9901-0acbec0722f8 service swift] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:26:51.057698 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3287d0cf-e281-41fc-9901-0acbec0722f8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:26:51.067104 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3287d0cf-e281-41fc-9901-0acbec0722f8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:26:51.068727 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3287d0cf-e281-41fc-9901-0acbec0722f8 service swift] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:26:51.077848 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1451/2876] 10.222.0.26 () {60 vars in 1384 bytes} [Thu Jul 14 09:26:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 14 09:26:52.416777 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f849b2a0-3131-4b37-acd7-d2ecd2d962f3 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:26:52.418463 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-f849b2a0-3131-4b37-acd7-d2ecd2d962f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:26:52.494375 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f849b2a0-3131-4b37-acd7-d2ecd2d962f3 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:26:52.499781 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c92dbd0-0635-47a9-8841-40327069e15d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:26:52.501157 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c92dbd0-0635-47a9-8841-40327069e15d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:26:52.501868 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c92dbd0-0635-47a9-8841-40327069e15d admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:26:52.502240 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c92dbd0-0635-47a9-8841-40327069e15d admin admin] PATH_INFO: `/v3/users/e9b3bbbc2a584649bf0078c977e7e919` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:26:52.503654 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c92dbd0-0635-47a9-8841-40327069e15d admin admin] RBAC: Authorizing `identity:delete_user(user_id=e9b3bbbc2a584649bf0078c977e7e919)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:26:52.505889 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c92dbd0-0635-47a9-8841-40327069e15d admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:26:52.580107 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-0c92dbd0-0635-47a9-8841-40327069e15d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e9b3bbbc2a584649bf0078c977e7e919'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:26:52.589473 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-0c92dbd0-0635-47a9-8841-40327069e15d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e9b3bbbc2a584649bf0078c977e7e919'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:26:52.590114 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-0c92dbd0-0635-47a9-8841-40327069e15d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e9b3bbbc2a584649bf0078c977e7e919'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:26:52.590859 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-0c92dbd0-0635-47a9-8841-40327069e15d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e9b3bbbc2a584649bf0078c977e7e919'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:26:52.600523 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-0c92dbd0-0635-47a9-8841-40327069e15d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e9b3bbbc2a584649bf0078c977e7e919'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:26:52.624093 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1426/2877] 10.222.0.26 () {58 vars in 1223 bytes} [Thu Jul 14 09:26:52 2022] DELETE /identity/v3/users/e9b3bbbc2a584649bf0078c977e7e919 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:26:52.628486 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dad8d6-043c-4b83-8f97-916a429283b5 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:26:52.628986 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dad8d6-043c-4b83-8f97-916a429283b5 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:26:52.629103 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dad8d6-043c-4b83-8f97-916a429283b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:26:52.649470 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-90dad8d6-043c-4b83-8f97-916a429283b5 None None] MFA Rules not processed for user `d00c29cc14a043099fbeda47a3389faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:26:52.650730 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-90dad8d6-043c-4b83-8f97-916a429283b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:26:52.670159 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:26:52.715868 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1452/2878] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jul 14 09:26:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 09:26:52.727123 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c92dbd0-0635-47a9-8841-40327069e15d admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:26:52.729054 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-0c92dbd0-0635-47a9-8841-40327069e15d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:26:52.753097 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:26:52.801856 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c92dbd0-0635-47a9-8841-40327069e15d admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:26:52.806053 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d1962f-f0f2-4a40-b6a2-9110fcc8bcc1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '952911c492444ffea4b211a24b564117', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:26:52.806872 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d1962f-f0f2-4a40-b6a2-9110fcc8bcc1 service neutron] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:26:52.807494 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d1962f-f0f2-4a40-b6a2-9110fcc8bcc1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:26:52.807657 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d1962f-f0f2-4a40-b6a2-9110fcc8bcc1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:26:52.818478 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d1962f-f0f2-4a40-b6a2-9110fcc8bcc1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:26:52.819967 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d1962f-f0f2-4a40-b6a2-9110fcc8bcc1 service neutron] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:26:52.830219 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1427/2879] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:26:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:26:53.160391 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dad8d6-043c-4b83-8f97-916a429283b5 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:26:53.161970 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-90dad8d6-043c-4b83-8f97-916a429283b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:26:53.175626 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dad8d6-043c-4b83-8f97-916a429283b5 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:26:53.181205 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10ace81-9361-4db3-b530-c59b335b713e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:26:53.182249 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10ace81-9361-4db3-b530-c59b335b713e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:26:53.183004 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10ace81-9361-4db3-b530-c59b335b713e admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:26:53.183004 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10ace81-9361-4db3-b530-c59b335b713e admin admin] PATH_INFO: `/v3/projects/8a75e8c5e3914518861e4353b03492fb` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:26:53.184269 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10ace81-9361-4db3-b530-c59b335b713e admin admin] RBAC: Authorizing `identity:delete_project(project_id=8a75e8c5e3914518861e4353b03492fb)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:26:53.186295 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10ace81-9361-4db3-b530-c59b335b713e admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:26:53.254753 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-c10ace81-9361-4db3-b530-c59b335b713e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8a75e8c5e3914518861e4353b03492fb', 'request_id': 'req-c10ace81-9361-4db3-b530-c59b335b713e'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:26:53.262760 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-c10ace81-9361-4db3-b530-c59b335b713e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8a75e8c5e3914518861e4353b03492fb', 'request_id': 'req-c10ace81-9361-4db3-b530-c59b335b713e'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:26:53.268901 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-c10ace81-9361-4db3-b530-c59b335b713e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8a75e8c5e3914518861e4353b03492fb', 'request_id': 'req-c10ace81-9361-4db3-b530-c59b335b713e'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:26:53.287238 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-c10ace81-9361-4db3-b530-c59b335b713e admin admin] The token cache is being invalidate because project 8a75e8c5e3914518861e4353b03492fb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80382) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 09:26:53.287238 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-c10ace81-9361-4db3-b530-c59b335b713e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:26:53.287615 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-c10ace81-9361-4db3-b530-c59b335b713e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:26:53.303219 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1453/2880] 10.222.0.26 () {58 vars in 1232 bytes} [Thu Jul 14 09:26:53 2022] DELETE /identity/v3/projects/8a75e8c5e3914518861e4353b03492fb => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:27:02.825487 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=19 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:27:03.280223 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=43,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:27:10.910799 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa12ee4-5350-4a18-ac26-1ce95f99c56d None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:27:10.911314 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa12ee4-5350-4a18-ac26-1ce95f99c56d None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:27:10.911508 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa12ee4-5350-4a18-ac26-1ce95f99c56d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:27:10.929815 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-3aa12ee4-5350-4a18-ac26-1ce95f99c56d None None] MFA Rules not processed for user `7d6267ede8d64e2abe1f4058a9797fb7`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:27:10.931103 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-3aa12ee4-5350-4a18-ac26-1ce95f99c56d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:27:10.988868 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1428/2881] 10.222.0.26 () {58 vars in 967 bytes} [Thu Jul 14 09:27:10 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 09:27:11.001354 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10ace81-9361-4db3-b530-c59b335b713e admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:27:11.003063 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-c10ace81-9361-4db3-b530-c59b335b713e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:27:11.086621 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10ace81-9361-4db3-b530-c59b335b713e admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:27:11.093085 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca39f887-277a-4be9-9919-ab6b1036abac service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5a43c80514b4a13af03c7f8ebcf7a5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:27:11.093741 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca39f887-277a-4be9-9919-ab6b1036abac service swift] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:27:11.094027 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca39f887-277a-4be9-9919-ab6b1036abac service swift] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:27:11.094303 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca39f887-277a-4be9-9919-ab6b1036abac service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:27:11.103569 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca39f887-277a-4be9-9919-ab6b1036abac service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:27:11.105191 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca39f887-277a-4be9-9919-ab6b1036abac service swift] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:27:11.114054 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1454/2882] 10.222.0.26 () {60 vars in 1384 bytes} [Thu Jul 14 09:27:11 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 14 09:27:13.074799 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa12ee4-5350-4a18-ac26-1ce95f99c56d None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:27:13.076418 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-3aa12ee4-5350-4a18-ac26-1ce95f99c56d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:27:13.152657 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa12ee4-5350-4a18-ac26-1ce95f99c56d None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:27:13.158106 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69b0fad-3748-4f27-b11a-703dd9c9cf38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:27:13.159240 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69b0fad-3748-4f27-b11a-703dd9c9cf38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:27:13.159586 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69b0fad-3748-4f27-b11a-703dd9c9cf38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:27:13.160052 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69b0fad-3748-4f27-b11a-703dd9c9cf38 admin admin] PATH_INFO: `/v3/users/9172fa641e5e4226b005d562e72d3cd8` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:27:13.161657 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69b0fad-3748-4f27-b11a-703dd9c9cf38 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9172fa641e5e4226b005d562e72d3cd8)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:27:13.164036 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69b0fad-3748-4f27-b11a-703dd9c9cf38 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:27:13.218125 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-b69b0fad-3748-4f27-b11a-703dd9c9cf38 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9172fa641e5e4226b005d562e72d3cd8'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:27:13.226247 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-b69b0fad-3748-4f27-b11a-703dd9c9cf38 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9172fa641e5e4226b005d562e72d3cd8'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:27:13.227092 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-b69b0fad-3748-4f27-b11a-703dd9c9cf38 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9172fa641e5e4226b005d562e72d3cd8'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:27:13.227774 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-b69b0fad-3748-4f27-b11a-703dd9c9cf38 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9172fa641e5e4226b005d562e72d3cd8'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:27:13.236155 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-b69b0fad-3748-4f27-b11a-703dd9c9cf38 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9172fa641e5e4226b005d562e72d3cd8'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:27:13.255696 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1429/2883] 10.222.0.26 () {58 vars in 1223 bytes} [Thu Jul 14 09:27:13 2022] DELETE /identity/v3/users/9172fa641e5e4226b005d562e72d3cd8 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:27:13.448006 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca39f887-277a-4be9-9919-ab6b1036abac service swift] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:27:13.462816 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca39f887-277a-4be9-9919-ab6b1036abac service swift] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:27:13.529011 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713bfe32-03a6-461b-91e5-15339ccbb8f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:27:13.530152 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713bfe32-03a6-461b-91e5-15339ccbb8f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:27:13.530975 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713bfe32-03a6-461b-91e5-15339ccbb8f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:27:13.531358 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713bfe32-03a6-461b-91e5-15339ccbb8f4 admin admin] PATH_INFO: `/v3/projects/250cb6b3464845868b20cf1f47ab031b` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:27:13.532679 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713bfe32-03a6-461b-91e5-15339ccbb8f4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=250cb6b3464845868b20cf1f47ab031b)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:27:13.534911 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713bfe32-03a6-461b-91e5-15339ccbb8f4 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:27:13.591269 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-713bfe32-03a6-461b-91e5-15339ccbb8f4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '250cb6b3464845868b20cf1f47ab031b', 'request_id': 'req-713bfe32-03a6-461b-91e5-15339ccbb8f4'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:27:13.599878 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-713bfe32-03a6-461b-91e5-15339ccbb8f4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '250cb6b3464845868b20cf1f47ab031b', 'request_id': 'req-713bfe32-03a6-461b-91e5-15339ccbb8f4'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:27:13.605412 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-713bfe32-03a6-461b-91e5-15339ccbb8f4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '250cb6b3464845868b20cf1f47ab031b', 'request_id': 'req-713bfe32-03a6-461b-91e5-15339ccbb8f4'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:27:13.619218 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-713bfe32-03a6-461b-91e5-15339ccbb8f4 admin admin] The token cache is being invalidate because project 250cb6b3464845868b20cf1f47ab031b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80382) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 09:27:13.619528 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-713bfe32-03a6-461b-91e5-15339ccbb8f4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:27:13.620232 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-713bfe32-03a6-461b-91e5-15339ccbb8f4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:27:13.621924 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1455/2884] 10.222.0.26 () {58 vars in 1232 bytes} [Thu Jul 14 09:27:13 2022] DELETE /identity/v3/projects/250cb6b3464845868b20cf1f47ab031b => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:27:22.084815 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69b0fad-3748-4f27-b11a-703dd9c9cf38 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:27:22.086375 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-b69b0fad-3748-4f27-b11a-703dd9c9cf38 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:27:22.116621 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:27:22.149618 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69b0fad-3748-4f27-b11a-703dd9c9cf38 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:27:22.198345 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987e4a68-4272-4709-ba63-53a3a4e2de80 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '952911c492444ffea4b211a24b564117', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:27:22.198991 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987e4a68-4272-4709-ba63-53a3a4e2de80 service neutron] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:27:22.199295 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987e4a68-4272-4709-ba63-53a3a4e2de80 service neutron] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:27:22.199595 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987e4a68-4272-4709-ba63-53a3a4e2de80 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:27:22.201596 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-987e4a68-4272-4709-ba63-53a3a4e2de80 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:27:22.265159 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987e4a68-4272-4709-ba63-53a3a4e2de80 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:27:22.266494 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987e4a68-4272-4709-ba63-53a3a4e2de80 service neutron] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:27:22.276788 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1430/2885] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:27:22 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:27:23.615785 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=83,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:27:32.272406 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=59 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:27:42.088526 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713bfe32-03a6-461b-91e5-15339ccbb8f4 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:27:42.090140 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-713bfe32-03a6-461b-91e5-15339ccbb8f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:27:42.104694 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713bfe32-03a6-461b-91e5-15339ccbb8f4 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:27:42.109137 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde6c7f0-3f86-4525-9f9d-e20eb2c28838 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '952911c492444ffea4b211a24b564117', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:27:42.109801 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde6c7f0-3f86-4525-9f9d-e20eb2c28838 service neutron] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:27:42.110063 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde6c7f0-3f86-4525-9f9d-e20eb2c28838 service neutron] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:27:42.110327 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde6c7f0-3f86-4525-9f9d-e20eb2c28838 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:27:42.112351 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-dde6c7f0-3f86-4525-9f9d-e20eb2c28838 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:27:42.192497 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde6c7f0-3f86-4525-9f9d-e20eb2c28838 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:27:42.194006 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde6c7f0-3f86-4525-9f9d-e20eb2c28838 service neutron] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:27:42.206526 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1456/2886] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:27:42 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:27:45.193621 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987e4a68-4272-4709-ba63-53a3a4e2de80 service neutron] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:27:45.195219 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-987e4a68-4272-4709-ba63-53a3a4e2de80 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:27:45.206971 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987e4a68-4272-4709-ba63-53a3a4e2de80 service neutron] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:27:45.211237 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a88686d-1162-46ec-bbeb-1f8d5803820e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '952911c492444ffea4b211a24b564117', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:27:45.211897 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a88686d-1162-46ec-bbeb-1f8d5803820e service neutron] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:27:45.212216 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a88686d-1162-46ec-bbeb-1f8d5803820e service neutron] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:27:45.212494 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a88686d-1162-46ec-bbeb-1f8d5803820e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:27:45.214448 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-7a88686d-1162-46ec-bbeb-1f8d5803820e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:27:45.290172 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a88686d-1162-46ec-bbeb-1f8d5803820e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:27:45.291683 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a88686d-1162-46ec-bbeb-1f8d5803820e service neutron] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:27:45.301956 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1431/2887] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:27:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:27:51.710214 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde6c7f0-3f86-4525-9f9d-e20eb2c28838 service neutron] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:27:51.711927 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-dde6c7f0-3f86-4525-9f9d-e20eb2c28838 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:27:51.726623 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde6c7f0-3f86-4525-9f9d-e20eb2c28838 service neutron] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:27:51.731773 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eafe6b0-50a6-4c21-a91b-e19628b3de82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:27:51.732685 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eafe6b0-50a6-4c21-a91b-e19628b3de82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:27:51.732972 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eafe6b0-50a6-4c21-a91b-e19628b3de82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:27:51.733257 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eafe6b0-50a6-4c21-a91b-e19628b3de82 admin admin] PATH_INFO: `/v3/users/e072ca213ed74cbda810a1fd971ae23d` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:27:51.734477 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eafe6b0-50a6-4c21-a91b-e19628b3de82 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e072ca213ed74cbda810a1fd971ae23d)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:27:51.736470 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eafe6b0-50a6-4c21-a91b-e19628b3de82 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:27:51.793586 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-1eafe6b0-50a6-4c21-a91b-e19628b3de82 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e072ca213ed74cbda810a1fd971ae23d'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:27:51.794263 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-1eafe6b0-50a6-4c21-a91b-e19628b3de82 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e072ca213ed74cbda810a1fd971ae23d'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:27:51.806402 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-1eafe6b0-50a6-4c21-a91b-e19628b3de82 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e072ca213ed74cbda810a1fd971ae23d'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:27:51.814483 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-1eafe6b0-50a6-4c21-a91b-e19628b3de82 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e072ca213ed74cbda810a1fd971ae23d'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:27:51.815154 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-1eafe6b0-50a6-4c21-a91b-e19628b3de82 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e072ca213ed74cbda810a1fd971ae23d'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:27:51.828168 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1457/2888] 10.222.0.26 () {58 vars in 1223 bytes} [Thu Jul 14 09:27:51 2022] DELETE /identity/v3/users/e072ca213ed74cbda810a1fd971ae23d => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:27:52.010998 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a88686d-1162-46ec-bbeb-1f8d5803820e service neutron] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:27:52.024052 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a88686d-1162-46ec-bbeb-1f8d5803820e service neutron] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:27:52.092898 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b918698a-351b-40f1-b036-6c64e795690b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:27:52.093700 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b918698a-351b-40f1-b036-6c64e795690b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:27:52.093981 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b918698a-351b-40f1-b036-6c64e795690b admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:27:52.094366 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b918698a-351b-40f1-b036-6c64e795690b admin admin] PATH_INFO: `/v3/projects/132e13bb6b1c413a8656f7a4ad816232` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:27:52.095703 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b918698a-351b-40f1-b036-6c64e795690b admin admin] RBAC: Authorizing `identity:delete_project(project_id=132e13bb6b1c413a8656f7a4ad816232)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:27:52.097653 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b918698a-351b-40f1-b036-6c64e795690b admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:27:52.149187 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-b918698a-351b-40f1-b036-6c64e795690b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '132e13bb6b1c413a8656f7a4ad816232', 'request_id': 'req-b918698a-351b-40f1-b036-6c64e795690b'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:27:52.156417 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-b918698a-351b-40f1-b036-6c64e795690b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '132e13bb6b1c413a8656f7a4ad816232', 'request_id': 'req-b918698a-351b-40f1-b036-6c64e795690b'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:27:52.162164 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-b918698a-351b-40f1-b036-6c64e795690b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '132e13bb6b1c413a8656f7a4ad816232', 'request_id': 'req-b918698a-351b-40f1-b036-6c64e795690b'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:27:52.176532 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-b918698a-351b-40f1-b036-6c64e795690b admin admin] The token cache is being invalidate because project 132e13bb6b1c413a8656f7a4ad816232 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80381) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 09:27:52.176819 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-b918698a-351b-40f1-b036-6c64e795690b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:27:52.177446 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-b918698a-351b-40f1-b036-6c64e795690b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:27:52.179058 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1432/2889] 10.222.0.26 () {58 vars in 1232 bytes} [Thu Jul 14 09:27:52 2022] DELETE /identity/v3/projects/132e13bb6b1c413a8656f7a4ad816232 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:27:52.183228 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eafe6b0-50a6-4c21-a91b-e19628b3de82 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:27:52.184924 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-1eafe6b0-50a6-4c21-a91b-e19628b3de82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:27:52.198887 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eafe6b0-50a6-4c21-a91b-e19628b3de82 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:27:52.251751 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:27:52.258426 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089724fd-b1f3-49d8-83fd-891d539c7654 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:27:52.259349 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089724fd-b1f3-49d8-83fd-891d539c7654 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:27:52.259637 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089724fd-b1f3-49d8-83fd-891d539c7654 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:27:52.259917 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089724fd-b1f3-49d8-83fd-891d539c7654 admin admin] PATH_INFO: `/v3/users/cdb088b3e09f4a80a58b9b12b164c48a` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:27:52.261281 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089724fd-b1f3-49d8-83fd-891d539c7654 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cdb088b3e09f4a80a58b9b12b164c48a)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:27:52.263660 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089724fd-b1f3-49d8-83fd-891d539c7654 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:27:52.314143 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-089724fd-b1f3-49d8-83fd-891d539c7654 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cdb088b3e09f4a80a58b9b12b164c48a'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:27:52.314781 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-089724fd-b1f3-49d8-83fd-891d539c7654 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cdb088b3e09f4a80a58b9b12b164c48a'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:27:52.326924 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-089724fd-b1f3-49d8-83fd-891d539c7654 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cdb088b3e09f4a80a58b9b12b164c48a'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:27:52.335887 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-089724fd-b1f3-49d8-83fd-891d539c7654 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cdb088b3e09f4a80a58b9b12b164c48a'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:27:52.336627 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-089724fd-b1f3-49d8-83fd-891d539c7654 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cdb088b3e09f4a80a58b9b12b164c48a'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:27:52.350214 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1458/2890] 10.222.0.26 () {58 vars in 1223 bytes} [Thu Jul 14 09:27:52 2022] DELETE /identity/v3/users/cdb088b3e09f4a80a58b9b12b164c48a => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:27:52.539286 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b918698a-351b-40f1-b036-6c64e795690b admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:27:52.540779 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-b918698a-351b-40f1-b036-6c64e795690b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:27:52.556430 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b918698a-351b-40f1-b036-6c64e795690b admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:27:52.577878 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:27:52.616389 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62064ca1-8e89-4188-a172-80d68b445d3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:27:52.617248 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62064ca1-8e89-4188-a172-80d68b445d3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:27:52.617635 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62064ca1-8e89-4188-a172-80d68b445d3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:27:52.617896 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62064ca1-8e89-4188-a172-80d68b445d3a admin admin] PATH_INFO: `/v3/projects/8f1ae6eac2dd48cba8bf16a163d69ee4` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:27:52.619107 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62064ca1-8e89-4188-a172-80d68b445d3a admin admin] RBAC: Authorizing `identity:delete_project(project_id=8f1ae6eac2dd48cba8bf16a163d69ee4)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:27:52.621087 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62064ca1-8e89-4188-a172-80d68b445d3a admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:27:52.674025 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-62064ca1-8e89-4188-a172-80d68b445d3a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8f1ae6eac2dd48cba8bf16a163d69ee4', 'request_id': 'req-62064ca1-8e89-4188-a172-80d68b445d3a'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:27:52.680196 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-62064ca1-8e89-4188-a172-80d68b445d3a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8f1ae6eac2dd48cba8bf16a163d69ee4', 'request_id': 'req-62064ca1-8e89-4188-a172-80d68b445d3a'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:27:52.685978 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-62064ca1-8e89-4188-a172-80d68b445d3a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8f1ae6eac2dd48cba8bf16a163d69ee4', 'request_id': 'req-62064ca1-8e89-4188-a172-80d68b445d3a'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:27:52.699610 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-62064ca1-8e89-4188-a172-80d68b445d3a admin admin] The token cache is being invalidate because project 8f1ae6eac2dd48cba8bf16a163d69ee4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80381) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 09:27:52.699897 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-62064ca1-8e89-4188-a172-80d68b445d3a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:27:52.700515 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-62064ca1-8e89-4188-a172-80d68b445d3a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:27:52.702114 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1433/2891] 10.222.0.26 () {58 vars in 1232 bytes} [Thu Jul 14 09:27:52 2022] DELETE /identity/v3/projects/8f1ae6eac2dd48cba8bf16a163d69ee4 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:28:02.346590 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=27,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:28:02.696736 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=40,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:28:55.583410 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089724fd-b1f3-49d8-83fd-891d539c7654 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:28:55.585283 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-089724fd-b1f3-49d8-83fd-891d539c7654 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:28:55.645747 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089724fd-b1f3-49d8-83fd-891d539c7654 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:28:55.697201 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f967fbf4-0939-432d-a5a2-56e0032fac01 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '952911c492444ffea4b211a24b564117', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:28:55.697851 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f967fbf4-0939-432d-a5a2-56e0032fac01 service neutron] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:28:55.698138 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f967fbf4-0939-432d-a5a2-56e0032fac01 service neutron] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:28:55.698414 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f967fbf4-0939-432d-a5a2-56e0032fac01 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:28:55.700596 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-f967fbf4-0939-432d-a5a2-56e0032fac01 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:28:55.710587 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f967fbf4-0939-432d-a5a2-56e0032fac01 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:28:55.712051 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f967fbf4-0939-432d-a5a2-56e0032fac01 service neutron] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:28:55.720258 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1459/2892] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:28:55 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:29:05.716757 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=46 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:29:21.658040 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62064ca1-8e89-4188-a172-80d68b445d3a admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:29:21.659890 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-62064ca1-8e89-4188-a172-80d68b445d3a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:29:21.674268 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62064ca1-8e89-4188-a172-80d68b445d3a admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:29:21.679290 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8608c3-e4a8-471c-b12e-44e18d6b8f0f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '952911c492444ffea4b211a24b564117', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:29:21.680092 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8608c3-e4a8-471c-b12e-44e18d6b8f0f service neutron] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:29:21.680368 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8608c3-e4a8-471c-b12e-44e18d6b8f0f service neutron] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:29:21.680663 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8608c3-e4a8-471c-b12e-44e18d6b8f0f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:29:21.682551 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-7f8608c3-e4a8-471c-b12e-44e18d6b8f0f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:29:21.766831 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8608c3-e4a8-471c-b12e-44e18d6b8f0f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:29:21.768285 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8608c3-e4a8-471c-b12e-44e18d6b8f0f service neutron] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:29:21.780428 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1434/2893] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:29:21 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:29:31.775424 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=34 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:29:33.938101 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f967fbf4-0939-432d-a5a2-56e0032fac01 service neutron] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:29:33.939729 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-f967fbf4-0939-432d-a5a2-56e0032fac01 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:29:33.950537 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f967fbf4-0939-432d-a5a2-56e0032fac01 service neutron] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:29:33.954980 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f785a6-6dc3-4c07-a270-9ddf00f75dd4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '952911c492444ffea4b211a24b564117', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:29:33.955744 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f785a6-6dc3-4c07-a270-9ddf00f75dd4 service neutron] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:29:33.956112 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f785a6-6dc3-4c07-a270-9ddf00f75dd4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:29:33.956449 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f785a6-6dc3-4c07-a270-9ddf00f75dd4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:29:33.958361 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-66f785a6-6dc3-4c07-a270-9ddf00f75dd4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:29:33.968552 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f785a6-6dc3-4c07-a270-9ddf00f75dd4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:29:33.970077 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f785a6-6dc3-4c07-a270-9ddf00f75dd4 service neutron] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:29:33.978541 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1460/2894] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:29:33 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:29:35.067411 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8608c3-e4a8-471c-b12e-44e18d6b8f0f service neutron] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:29:35.077032 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8608c3-e4a8-471c-b12e-44e18d6b8f0f service neutron] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:29:35.081516 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201953ce-7685-4149-b81a-7f5faa6646aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '952911c492444ffea4b211a24b564117', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:29:35.082166 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201953ce-7685-4149-b81a-7f5faa6646aa service neutron] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:29:35.082571 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201953ce-7685-4149-b81a-7f5faa6646aa service neutron] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:29:35.082880 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201953ce-7685-4149-b81a-7f5faa6646aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:29:35.084872 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-201953ce-7685-4149-b81a-7f5faa6646aa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:29:35.102082 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201953ce-7685-4149-b81a-7f5faa6646aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:29:35.104072 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201953ce-7685-4149-b81a-7f5faa6646aa service neutron] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:29:35.115770 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1435/2895] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:29:35 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:29:35.210199 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f785a6-6dc3-4c07-a270-9ddf00f75dd4 service neutron] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:29:35.211848 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-66f785a6-6dc3-4c07-a270-9ddf00f75dd4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:29:35.287467 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f785a6-6dc3-4c07-a270-9ddf00f75dd4 service neutron] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:29:35.292504 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03111da-e389-4783-ad28-823d95bc08bb service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '355684f52e074ad9bfa7fd90bcae6caf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:29:35.293178 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03111da-e389-4783-ad28-823d95bc08bb service placement] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:29:35.293460 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03111da-e389-4783-ad28-823d95bc08bb service placement] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:29:35.293735 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03111da-e389-4783-ad28-823d95bc08bb service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:29:35.295633 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-e03111da-e389-4783-ad28-823d95bc08bb service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:29:35.306892 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03111da-e389-4783-ad28-823d95bc08bb service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:29:35.308534 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03111da-e389-4783-ad28-823d95bc08bb service placement] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:29:35.318244 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1461/2896] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:29:35 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:29:36.454019 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201953ce-7685-4149-b81a-7f5faa6646aa service neutron] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:29:36.455838 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-201953ce-7685-4149-b81a-7f5faa6646aa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:29:36.476885 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201953ce-7685-4149-b81a-7f5faa6646aa service neutron] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:29:36.484155 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a1545c-3ca4-46e4-963a-9d2bb1451c3a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '538fff3c5a2b487eb44ae7c1a1198102', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:29:36.484959 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a1545c-3ca4-46e4-963a-9d2bb1451c3a service nova] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:29:36.485431 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a1545c-3ca4-46e4-963a-9d2bb1451c3a service nova] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:29:36.486070 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a1545c-3ca4-46e4-963a-9d2bb1451c3a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:29:36.488396 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-e4a1545c-3ca4-46e4-963a-9d2bb1451c3a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:29:36.502410 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a1545c-3ca4-46e4-963a-9d2bb1451c3a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:29:36.504108 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a1545c-3ca4-46e4-963a-9d2bb1451c3a service nova] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:29:36.527364 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1436/2897] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:29:36 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:29:45.314132 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=38 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:29:46.510832 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=12 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:31:03.847598 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03111da-e389-4783-ad28-823d95bc08bb service placement] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:31:03.849081 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-e03111da-e389-4783-ad28-823d95bc08bb service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:31:03.866552 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03111da-e389-4783-ad28-823d95bc08bb service placement] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:31:03.872648 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1eb2785-f31c-4363-a13f-42059a076af5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '538fff3c5a2b487eb44ae7c1a1198102', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:31:03.873339 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1eb2785-f31c-4363-a13f-42059a076af5 service nova] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:31:03.873626 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1eb2785-f31c-4363-a13f-42059a076af5 service nova] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:31:03.873905 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1eb2785-f31c-4363-a13f-42059a076af5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:31:03.875832 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-b1eb2785-f31c-4363-a13f-42059a076af5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:31:03.889606 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1eb2785-f31c-4363-a13f-42059a076af5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:31:03.891405 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1eb2785-f31c-4363-a13f-42059a076af5 service nova] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:31:03.914701 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1462/2898] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:31:03 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:31:10.858221 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a1545c-3ca4-46e4-963a-9d2bb1451c3a service nova] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:31:10.859898 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-e4a1545c-3ca4-46e4-963a-9d2bb1451c3a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:31:10.874292 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a1545c-3ca4-46e4-963a-9d2bb1451c3a service nova] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:31:10.879394 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa291f6-f155-4f64-8529-d3c4ca5d684d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '355684f52e074ad9bfa7fd90bcae6caf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:31:10.880078 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa291f6-f155-4f64-8529-d3c4ca5d684d service placement] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:31:10.880554 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa291f6-f155-4f64-8529-d3c4ca5d684d service placement] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:31:10.880842 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa291f6-f155-4f64-8529-d3c4ca5d684d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:31:10.882615 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-baa291f6-f155-4f64-8529-d3c4ca5d684d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:31:10.896214 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa291f6-f155-4f64-8529-d3c4ca5d684d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:31:10.898011 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa291f6-f155-4f64-8529-d3c4ca5d684d service placement] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:31:10.920590 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1437/2899] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:31:10 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:31:13.898757 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=6 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:31:14.049886 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1eb2785-f31c-4363-a13f-42059a076af5 service nova] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:31:14.051042 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-b1eb2785-f31c-4363-a13f-42059a076af5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:31:14.126741 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1eb2785-f31c-4363-a13f-42059a076af5 service nova] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:31:14.131470 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0773fad4-8ba4-4b10-a578-4087486a1210 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:31:14.132359 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0773fad4-8ba4-4b10-a578-4087486a1210 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:31:14.132615 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0773fad4-8ba4-4b10-a578-4087486a1210 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:31:14.132911 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0773fad4-8ba4-4b10-a578-4087486a1210 admin admin] PATH_INFO: `/v3/users/56bd87cf833847cc9c8b5e78348de6fc` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:31:14.134104 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0773fad4-8ba4-4b10-a578-4087486a1210 admin admin] RBAC: Authorizing `identity:delete_user(user_id=56bd87cf833847cc9c8b5e78348de6fc)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:31:14.136071 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0773fad4-8ba4-4b10-a578-4087486a1210 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:31:14.187070 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-0773fad4-8ba4-4b10-a578-4087486a1210 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '56bd87cf833847cc9c8b5e78348de6fc'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:31:14.187854 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-0773fad4-8ba4-4b10-a578-4087486a1210 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '56bd87cf833847cc9c8b5e78348de6fc'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:31:14.201293 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-0773fad4-8ba4-4b10-a578-4087486a1210 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '56bd87cf833847cc9c8b5e78348de6fc'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:31:14.210236 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-0773fad4-8ba4-4b10-a578-4087486a1210 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '56bd87cf833847cc9c8b5e78348de6fc'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:31:14.210840 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-0773fad4-8ba4-4b10-a578-4087486a1210 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '56bd87cf833847cc9c8b5e78348de6fc'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:31:14.223449 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1463/2900] 10.222.0.26 () {58 vars in 1223 bytes} [Thu Jul 14 09:31:14 2022] DELETE /identity/v3/users/56bd87cf833847cc9c8b5e78348de6fc => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:31:14.410297 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa291f6-f155-4f64-8529-d3c4ca5d684d service placement] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:31:14.426920 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa291f6-f155-4f64-8529-d3c4ca5d684d service placement] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:31:14.488192 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da16df8-5c90-41e3-8884-dda43942a675 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:31:14.489090 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da16df8-5c90-41e3-8884-dda43942a675 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:31:14.489347 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da16df8-5c90-41e3-8884-dda43942a675 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:31:14.489745 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da16df8-5c90-41e3-8884-dda43942a675 admin admin] PATH_INFO: `/v3/projects/8e3d77e253bd4167b4be5c923e026228` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:31:14.490827 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da16df8-5c90-41e3-8884-dda43942a675 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8e3d77e253bd4167b4be5c923e026228)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:31:14.493614 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da16df8-5c90-41e3-8884-dda43942a675 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:31:14.549668 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-0da16df8-5c90-41e3-8884-dda43942a675 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8e3d77e253bd4167b4be5c923e026228', 'request_id': 'req-0da16df8-5c90-41e3-8884-dda43942a675'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:31:14.556619 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-0da16df8-5c90-41e3-8884-dda43942a675 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8e3d77e253bd4167b4be5c923e026228', 'request_id': 'req-0da16df8-5c90-41e3-8884-dda43942a675'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:31:14.563019 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-0da16df8-5c90-41e3-8884-dda43942a675 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8e3d77e253bd4167b4be5c923e026228', 'request_id': 'req-0da16df8-5c90-41e3-8884-dda43942a675'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:31:14.577339 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-0da16df8-5c90-41e3-8884-dda43942a675 admin admin] The token cache is being invalidate because project 8e3d77e253bd4167b4be5c923e026228 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80381) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 09:31:14.577592 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-0da16df8-5c90-41e3-8884-dda43942a675 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:31:14.578246 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-0da16df8-5c90-41e3-8884-dda43942a675 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:31:14.579908 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1438/2901] 10.222.0.26 () {58 vars in 1232 bytes} [Thu Jul 14 09:31:14 2022] DELETE /identity/v3/projects/8e3d77e253bd4167b4be5c923e026228 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:31:14.584102 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0773fad4-8ba4-4b10-a578-4087486a1210 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:31:14.585587 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-0773fad4-8ba4-4b10-a578-4087486a1210 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:31:14.599426 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0773fad4-8ba4-4b10-a578-4087486a1210 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:31:14.667830 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ffb5e3-f679-4e3a-b088-88bf350bbea2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:31:14.668859 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ffb5e3-f679-4e3a-b088-88bf350bbea2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:31:14.669200 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ffb5e3-f679-4e3a-b088-88bf350bbea2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:31:14.669528 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ffb5e3-f679-4e3a-b088-88bf350bbea2 admin admin] PATH_INFO: `/v3/users/02c93dbd9c3c4e51ba19072293cab44b` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:31:14.671002 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ffb5e3-f679-4e3a-b088-88bf350bbea2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=02c93dbd9c3c4e51ba19072293cab44b)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:31:14.673022 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ffb5e3-f679-4e3a-b088-88bf350bbea2 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:31:14.696236 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:31:14.725447 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-90ffb5e3-f679-4e3a-b088-88bf350bbea2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '02c93dbd9c3c4e51ba19072293cab44b'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:31:14.726212 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-90ffb5e3-f679-4e3a-b088-88bf350bbea2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '02c93dbd9c3c4e51ba19072293cab44b'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:31:14.739081 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-90ffb5e3-f679-4e3a-b088-88bf350bbea2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '02c93dbd9c3c4e51ba19072293cab44b'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:31:14.747579 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-90ffb5e3-f679-4e3a-b088-88bf350bbea2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '02c93dbd9c3c4e51ba19072293cab44b'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:31:14.748310 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-90ffb5e3-f679-4e3a-b088-88bf350bbea2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '02c93dbd9c3c4e51ba19072293cab44b'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:31:14.761655 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1464/2902] 10.222.0.26 () {58 vars in 1223 bytes} [Thu Jul 14 09:31:14 2022] DELETE /identity/v3/users/02c93dbd9c3c4e51ba19072293cab44b => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 09:31:14.948187 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da16df8-5c90-41e3-8884-dda43942a675 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:31:14.949873 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-0da16df8-5c90-41e3-8884-dda43942a675 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:31:14.963878 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da16df8-5c90-41e3-8884-dda43942a675 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:31:15.034999 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a767bbbe-686b-47d9-b830-395cc29b24d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00c29cc14a043099fbeda47a3389faa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fd144ddf49f41d58d6674bed415082b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:31:15.035921 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a767bbbe-686b-47d9-b830-395cc29b24d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:31:15.036199 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a767bbbe-686b-47d9-b830-395cc29b24d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:31:15.036472 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a767bbbe-686b-47d9-b830-395cc29b24d1 admin admin] PATH_INFO: `/v3/projects/00aed77165b04cdd9becc90c19248929` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:31:15.037846 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a767bbbe-686b-47d9-b830-395cc29b24d1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=00aed77165b04cdd9becc90c19248929)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:31:15.039939 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a767bbbe-686b-47d9-b830-395cc29b24d1 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:31:15.063888 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:31:15.089679 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-a767bbbe-686b-47d9-b830-395cc29b24d1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '00aed77165b04cdd9becc90c19248929', 'request_id': 'req-a767bbbe-686b-47d9-b830-395cc29b24d1'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:31:15.096485 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-a767bbbe-686b-47d9-b830-395cc29b24d1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '00aed77165b04cdd9becc90c19248929', 'request_id': 'req-a767bbbe-686b-47d9-b830-395cc29b24d1'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:31:15.103987 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-a767bbbe-686b-47d9-b830-395cc29b24d1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '00aed77165b04cdd9becc90c19248929', 'request_id': 'req-a767bbbe-686b-47d9-b830-395cc29b24d1'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:31:15.117859 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-a767bbbe-686b-47d9-b830-395cc29b24d1 admin admin] The token cache is being invalidate because project 00aed77165b04cdd9becc90c19248929 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80381) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 09:31:15.118148 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-a767bbbe-686b-47d9-b830-395cc29b24d1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:31:15.118779 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-a767bbbe-686b-47d9-b830-395cc29b24d1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 09:31:15.120400 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1439/2903] 10.222.0.26 () {58 vars in 1232 bytes} [Thu Jul 14 09:31:14 2022] DELETE /identity/v3/projects/00aed77165b04cdd9becc90c19248929 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 09:31:24.758373 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:DELETE=7,keystone:SELECT=20,keystone:INSERT=1 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:31:25.114692 dv-cinder-849626-4-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=16,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:31:56.183471 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ffb5e3-f679-4e3a-b088-88bf350bbea2 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:31:56.185170 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-90ffb5e3-f679-4e3a-b088-88bf350bbea2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:31:56.258568 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ffb5e3-f679-4e3a-b088-88bf350bbea2 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:31:56.320742 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0fff58-5a5d-406b-bdcb-06739ab83fe4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '355684f52e074ad9bfa7fd90bcae6caf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33bf210d015847439655b9943d1b0261', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:31:56.321386 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0fff58-5a5d-406b-bdcb-06739ab83fe4 service placement] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:31:56.321663 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0fff58-5a5d-406b-bdcb-06739ab83fe4 service placement] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:31:56.321941 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0fff58-5a5d-406b-bdcb-06739ab83fe4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:31:56.324017 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-8b0fff58-5a5d-406b-bdcb-06739ab83fe4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:31:56.336387 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0fff58-5a5d-406b-bdcb-06739ab83fe4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:31:56.337990 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0fff58-5a5d-406b-bdcb-06739ab83fe4 service placement] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:31:56.349016 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1465/2904] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jul 14 09:31:56 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 09:32:06.344099 dv-cinder-849626-4-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=58 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}